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ABSTRACT 

This document presents the proceedings of the International 
Academy for Information Management’s International Conference on Informatics 
Education and Research (ICIER), held December 14-16, 2001 in New Orleans, 
Louisiana. The keynote address was given by Joseph A. Grace, Jr., founding 
and current President of the Louisiana Technology Council. Presentations 
include: "Who do we need to motivate? Toward an Integrative Model of E- 
education" (Celia Romm, Arik Ragowsky) ; "Technology for Managing Professional 
Knowledge of Future Software Developers" (Aybuke Aurum) ; "Student Learning 
Styles & Distance Learning" (Raymond Papp) ; "From Distance Education to 
Flexible Learning" (Tim S Roberts); "Application Of A Model For Improving ICT 
Management: A Case Study" (Gregory Wanyembi); "RoboCup: Multi-disciplinary 
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Gender Issues In The Computer and Information Technology Curriculum" (Mary J. 
Granger, Joyce Currie Little, Susan K. Lippert) ; "Effective Case Study 
Methodologies In The Management Of IT Courses" (James R. Buffington, Jeffrey 
S .• Harper); "Teaching With Online Case Studies: Implementation And Evaluation 
Issues" (Roberto Vinaja, Mahesh S. Raisinghani) ; "Using the Web as a 
Strategic Resource:- An Applied Classroom Exercise" (Kathleen M. Wright, Mary 
J. Granger) ; "A Hierarchy of Needs for a Virtual Class" (Catherine Beise, 

Judy Wynekoop) ; "Development Of An E-Commerce Model Curriculum" (Dwayne 
Whitten, Charlotte Stephens) ; "Aligning E-Business Programs with Industry 
Needs" (Keng Siau, Sid Davis, Kumar Dhenuvakonda ) ; "Preparing For Information 
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Strategy to Overcome Challenges in Teaching an Introductory MIS 
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Survey of Database Curricula Using Web-Posted Syllabi" (Joseph T. Harder); 
"Requiring Students To Bring Computers To Campus: Are Universities Achieving 
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Adopt Internet-Based Health Applications: A Field Study Utilizing The 
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"An Exploratory Study Of The Structure And Activities Of MIS Advisory 
Boards" (James R. Buffington, Dennis Bialaszewski , Janejira Sutanonpaiboon) ; 
"Analyzing the Factors that Affect Intention to Enter a Post Baccalaureate 
Business Education Program" (Richard McCarthy, George Claffey) ; "Adopting 
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International Academy for Information Management’s 

Imtematiomal Conference ©e Informatics Education & Research 

December 14-16, 2001 
New Orleans, Louisiana 

Fr i da y December 14 

Opening Reception (6:00 pm - 8:30 pm) 

Saturday December 15 

Keynote Address (8:00 am - 9:30 am) S es sion s IA - ID (9:45 am - 10:45 am) 

Sessions 2 A - 2D (1 1 :00 am - 12:00 am) Sessions 3A - 3D (1 :45 pm - 2:45 pm) 

Sessions 4A - 4D (3 :00 pm - 4 :00 pm) Sessi o n s 5A - 5D (4 : 1 5 pm - 5 : 1 5 pm) 

Sunday Decembe r 16 

Sessions 6A - 6D (9:00 am - 10:00 am) Sessions 7A- 7D (10:30 am - 12:00 pm) 

Session 8 - Joint 1A1M/ICIS (12:30 pm — 1 :30 pm) 



Au t ho r C ontact I n f orm at io n 

Friday, December 14 

I AIM Board Meet ing (9:00 A.M. - 6:00 P.M.) Mayor’s Chambers 
Opening Reception (6:00 - 8:30 P.M.) Explorers Room 



Saturday, December 15 

Breakfast Buffet (7:30 - 8:30) Bayou Lobby 



Keynote Address (8:00 - 9:30) 

(Bayou 1 & 3) 

Joseph A. Grace, Jr. is the founding and current President of the Louisian a Technology Council . 
Joe is a 1980 graduate of the United States Naval Academy and a former Nuclear Submarine 
Officer. He comes from a well-established Louisiana family and spent most of his early years 
overseas where his father was in the Diplomatic Corps as a Naval Attache. After leaving the Navy 
in 1988, Joe toured as a musician and recording artist before "getting a real job with a fortune 
500 Electronics firm in California, where he held various positions in sales and sales management. 
He came home to Louisiana in November of 1994 to assume the position as President and develop 
the New Orleans Technology Council. He is fully committed to transforming the Louisiana 
Technology industry and serves on multiple Boards and committees throughout the State. 



Sessions 1A-1B (9:45 - 10:45) 

1A - Pedagogical Issues I (Bayou 1) 

SESSION CHAIR: Diane Fischer, Dowling College 

"Who do we need to motivate? Toward an Integrative Model of E-education" 

Celia Romm, Central Queensland University, and Arik Ragowsky, Wayne State University. 
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" Technology for Man a ging Profess i onal Knowledge of Future Software Developers" 

Aybuke Aurum, University of New South Wales. 

IB - Internet/Distance Education I (Bayou 2) 

SESSION CHAIR: Mary Brabston, The University of Manitoba 

“Student Learnin g Sty l es & Dista n ce Learning” 

Raymond Papp, University of Tampa. 

“From Distance Education t o Flexib le Le arnin g” 

Tim S Roberts, Central Queensland University. 

1C - Curriculum Issues I (Bayou 3) 

SESSION CHAIR: Jim Buffington, Indiana State University 

“Applic a tio n Of A Mode l Fo r I m proving I CT Management: A Case Study” 

Gregory Wanyembi, Delft University of Technology. 

“RoboCup: Multi-discip l inar y S enior Design Projec t ” 

Kevin Lee Elder, Lonnie R. Welch, David M. Chelberg and Douglas A. Lawrence, Jae Y. Lew and 
Robert L. Williams II. 

ID - PANEL Presentation (Bayou 4) 

SESSION CHAIR: Mary Granger, George Washington University 

“C ul tural and G en der Is s ues In Th e Computer and Information Technolo g y Curriculum” 

Mary J. Granger, George Washington University, Joyce Currie Little, Towson University, and 
Susan K. Lippert, George Washington University. 



Sessions 2A-2D (1.1:00 - 12:00) 

2A - Peda go gical Issues I I (Bayou 1) 

SESSION CHAIR: Max Burns, Georgia Southern University 
"Effective Ca se S t udy Methodo logie s In The Management Of IT Courses ” 

James R. Buffington, and Jeffrey S. Harper, Indiana State University. 

“Te ac hing With Onli ne C ase Studi es: I mp l ementat i on And Ev a l uation Iss ue s” 

Roberto Vinaja, University of Texas Pan American, and Mahesh S. Raisinghani, University of 
Dallas. 



2B - Internet/Distance Education II (Bayou 2) 

SESSION CHAIR: Cindy Randall, Georgia Southern University 

“Using the W eb as a S tr ateg i c Resource: An A pplied C lassr oom Ex er cis e ” 

Kathleen M. Wright, Salisbury State University, and Mary J. Granger, George Washington 
University. 

“A Hierarchy of Needs for a Virtual Class” 

Catherine Beise, and Judy Wynekoop, Florida Gulf Coast University. 
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2C - Curriculum Issues II (Bayou 3) 

SESSION CHAIR: Celia Romm, Central Queensland University 



“Development Of An E-Commerce Model Curriculum” 

Dwayne Whitten, Ouachita Baptist University, and Charlotte Stephens, Louisiana Tech 
University. 

“Aligning E-Busine s s Prog r am s with Industry Needs” 

Keng Siau, Sid Davis, and Kumar Dhenuvakonda, University of Nebraska -Lincoln. 

2D - PANEL Presentation (Bayou 4) 

SESSION CHAIR: Doris Lidtke, Towson State 

“Preparing For Inf ormation S ys tems Ac c reditation” 

Doris K. Lidtke, Towson University, John Gorgone, Bentley College, and David Feinstein, 
University of South Alabama. 



Luncheon & Annual Meetin g (12:00 - 1:30) Bailey Room 



Sessions 3A-3D (1:45 - 2:45) 

3A — Pedag o gical Issues III (Bayou 1) 

SESSION CHAIR: Geoffrey Dick, University of New South Wales 

“A Strategy to Overcome Challenges in Teaching an Introductory MIS Cour sed 

Aileen Cater-Steel, Ee Kuan Low, and Rohan Genrich, University of Southern Queensland, 

Australia. 

“ T e aching Data War eho us ing to G r adua te St ud ents - A Cas e St udy!! 

MC Matthee, HL Viktor, J Pretorius and E Venter, University of Pretoria. 

3B - Internet/Distance Education III (Bayou 2) 

SESSION CHAIR: Tom Schambach, Illinois State University 

“End User Su pp ort as a R e sult of Computing Educat i on ” 

Susan E. Yager and John F. Schrage, Southern Illinois University. 

“Teaching the evaluation of Electronic Commerce sites using HCI techniques” 

Carina de Villiers, University of Pretoria. 

3C - Curriculum Issues III (Bayou 3) 

SESSION CHAIR: Charlotte Stephens, Columbus State University 

“A Survey of Database Curricula Usin g Web-Posted Syllabi” 

Joseph T. Harder, Indiana State University. 

“Requirin g Students To Bring Computers To Campus: Are Universities Achieving Their Goals?” 
Cindy H. Randall, and Barbara A. Price, Georgia Southern University. 
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3D - Best Paper Finalists I (Bayou 4) 

SESSION CHAIR: Tom Pencek, Meredith College 

“Im plementin g a ‘Third Place’ for MIS Students: A Preliminar y Report^ 

Lori Baker-Eveleth, Washington State University, Suprateek Sarker, Washington State 
University, and Daniel Eveleth, University of Idaho. 

“Comparing the Expectations of Inform a t i on Technology Students to the E xpectatjoniLoLG^ 
Pat Hodd, and Brian Dobing, University of Lethbridge. 



Sessions 4A-4D (3:00 - 4:00) 

4A - Pedagogical Issues IV (Bayou 1) 

SESSION CHAIR: Camille Rogers, Georgia Southern University 
“Stude nts’ Percep t ion o f P BL Assessmen t” 

Ahmad Ghafarian, North Georgia College & State University & Willie Yip, The Hong Kong 
Polytechnic University. 

“Com p arison of Live Versus Pa per-Based Assessment in Com puter Ap plication Courses” 

Nicole B. Koppel, and Kimberly A. Killmer, Montclair State University. 

4B - Internet/Distance Education IV (Bayou 2) 

SESSION CHAIR: Susan Lippert, Drexel University 

“ T el e m ati c Edu cat ion i n I nf orm atics: A cas e stu d y of the pit fall s a nd the c h allen g e s” 

A.C (Awie) Leonard, and WM (Wayne) Motha. University of Pretoria. 

“Predictin g Ph ysicians’ Inte ntions To A do pt Internet-Based Health Ap plications: A Field Stud y Utilizin g 
The Extended Technology Acceptance Model” 

Sonja Wiley-Patton, and William Chismar, University of Hawaii Manoa. 

4C - Curriculum Issues IV (Bayou 3) 

SESSION CHAIR: Ali Jenzarli, University of Tampa 

“A n Expl o ratory Study Of The Structu re An d Ac tivities Of MIS Advisory Boards” 

James R. Buffington, Dennis Bialaszewski, and Janejira Sutanonpaiboon, Indiana State 
University. 

“Analyzing the Factors that Affect Intention to Enter a Post Baccalaureate Business Education Program” 
Richard McCarthy, Quinnipiac University, and George Claffey, Central Connecticut State 
University. 

4D - Best Paper Finalists II (Bayou 4) 

SESSION CHAIR: James Coakley, Oregon State University 

“Adopting Distance Education - What do the Students Think?” 

Geoffrey N. Dick, University of New South Wales, Tom Case, Georgia Southern University, and 
O. Maxie Burns, Georgia Southern University. 

“SERVQUAL-Based Measurement of Student Satisfaction with Classroom Instructional Technologies: A 
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2001 Update” 

Betty A. Kleen and L. Wayne Shell, Nicholls State University. 



Sessions 5A-5B (4:15 - 5:15) 

5A - Pedagogical Issues V (Bayou 1) 

SESSION CHAIR: Richard McCarthy, Quinnipiac University 

"A Fram e w or k for Contro lli n g Ch eating in I S Ed u cat i on” 

Janies R. Coakley and Craig K. Tyran, Oregon State University. 

“Pre-traininq Motivation to Learn: Construct Exploration in an Information Systems Context” 

Cynthia LeRouge, University of South Florida, Creggan Gjestland, University of Alabama, and 
Ellis Blanton, University of South Florida. 

5B - Internet/Distance Education V (Bayou 2) 

SESSION CHAIR: Marianne D'Onofrio, Central Connecticut State University 
‘‘ A Finance Ap pli cat i on O f I nform at ion T echno logy” 

Thomas Pencek, Meredith College, and Lisa MacLean Wentworth Institute of Technology. 

“ Tutorial for A Framework to Introduce the Extensi bl e Mar kup Lang uage (XML) into Informa ti on Syst ems 
C oursework” 

Meg Murray and Stephanie Cupp, Kennesaw State University. 

5C - Curricu l um Issues V (Bayou 3) 

SESSION CHAIR: Stephanie Huneycutt, Christopher Newport University 

“Usin g IT to Teach Proj ect Ma n ageme nt i n t he MBA Core” 

Ali Jenzarli, University of Tampa. 

“E-Bus i ness And The Glob a l A da ptat i on Of MBA Pro grams” 

Harald Mahrer and Roman Brandtweiner, Vienna University of Economics and Business 
Administration. 



5D - PANEL: Distance Learning (Bayou 4) 

SESSION CHAIR: Raymond Papp, University of Tampa 

“Wha t Makes D ist r i buted L e arning E f fe c tive ? ” 

Tom Case Georgia, Southern University, Diane Fischer, Dowling College, Muhammadou M.O. 
Kah, Rutgers University, Carol Okolica, Dowling College, and Raymond Papp, University of 
Tampa. 



Saturday Post-Session (5:30-6:00) 

Bayou 1 

“Al pha lota Mu presentation” (Learn about the new Information Systems & Technology honor society) 
James R. Buffington & Brent L. Bacon, Indiana State University. 



http://www.iaim.org/ICIER2001/icier2001 .htm 



7 



10/10/2002 



Sunday, December 16 

Breakfast Buffet (8:00 - 9:00) Bayou Lobby 



Sessions 6A-6D (9:00 — 10:00) 

6A - Programming Issues (Bayou 1) 

SESSION CHAIR: Behrooz Seyed-Abbassi, University of North Florida 

“A Comparison of Business and Game Projects For Intermediate Programming Classes” 

Brian Dobing, University of Lethbridge, and David Erbach, Purdue University - Fort Wayne. 

“Prog ram mingLanguages forTeachingObject-Orientation” 

Michael A. Eierman, University of Wisconsin Oshkosh . 



6B - Knowledge Sharing (Bayou 2) 

SESSION CHAIR: Marianne D’Onofrio, Central Connecticut State University 

“ Know ledge sharing tool s and the facilita ti on of student team performance in IS .courses.” 
Seokwoo Song and Thomas Case, Georgia Southern University. 

“A Commit t ee-Based Model f or Supportin g O r ganizational Knowledge Manag ement ” 

Les L. Miller* and Sree Nilakanta, Iowa State University. 



6C - Collaborative Learning (Bayou 3) 

SESSION CHAIR: Tom Case, Georgia Southern University 

“Collaborative Learning and Group Assessment: Introducing the Capitalist and Socialist Paradigms" 
Tim S Roberts, Central Queensland University. 

“A Met a-Analysis of Research on Student Team Effe ctivene ss: A Pro posed Applicatio n of Pha sed 
Interventions” 

Charlotte S. Stephens, Louisiana Tech University. 

6D - PANEL Presentation (Bayou 4) 

SESSION CHAIR: Patricia Sendall, Merrimack College 

“ A Com p a r ison of E-Commerce and Information Sys t ems Job T ype s: Is There a Difference?” 
Patricia Sendall, Merrimack College, and Herbert E. Longenecker, Jr., University of South 
Alabama. 



Sessions 7A-7C (10:30 - 12:00) 

7A - Social Issues (Bayou 1) 

SESSION CHAIR: Seokwoo Song, Georgia Southern University 
“Social Issues and e-Business” 

Candance Deans, Thunderbird -- the American Graduate School of International Management 
Paul Ulrickson, Consultant/Relationship Coach 
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7B - Potpourri Issues (Bayou 2) 

SESSION CHAIR: Betty Kleen, Nicholls State University 



“Enterprise Resource Planning Systems: Factors Affecting Implementation” 

Carol Okolica, Dowling College. 

“Interested in Improving: Your Teaching . Student Placeme n t . Op portunities for A pplied Researctb 
Practical Knowled ge . Professional Network and External Financial Out Reach?” 

Rick Weible, Marshall University. 

“Developing a R o bust S ystem for Effective Teamwork on Lengthy, Complex Tasks: An Empirical 
Exp l o r a ti on of Int er v en t ion s t o In creas e Team Effe cti ven e ss ” 

Charlotte S. Stephens, Louisiana Tech University, and Martha E. Myers, Kennesaw State 
University. 



7C- Table Topics I (Bayou 3) 

SESSION CHAIR: Brian Dobing, University of Lethbridge 

“The MFC Framework as a Case Study on Effective Ob je c t-Oriented Design” 

Yonglei Tao, Grand Valley State University. 

"Tria ls and Tribulations o f U pdating an IS Curriculum” 

Behrooz Seyed-Abbassi, University of North Florida. 

“The Employment Out Loo k for I nformatio n Techn olog y Workers: A Demand Forecast” 

Rick Weible, Marshall University. 

“An A p pr o a ch T o Te aching Information Technology And Syste m Ar c hitectures” 

Annette Lerine Steenkamp, University of Detroit Mercy. 

“The Internet and Intellectual Property Rights: What Business Managers and Business Maj or s Need to 
Know” 

Carol Okolica, Dowling College and Laurette D. Mulry, Legal Aid Society. 

7D - Table Topics II (Bayou 4) 

SESSION CHAIR: Seokwoo Song, Georgia Southern University 
“S ystems Dev elo pment Pract i c e s: Circa 200 1” 

Thomas Schambach and Kent Walstrom, Illinois State University. 

“Changing Trends In Corporate Education: Impacting University Education?” 

Abdus Sami Nagi and Mary J. Granger, George Washington University. 

“Be Prepared for the Working Environment - Teamwork for Information Systems Students” 

Gys le Roux, Pieter Joubert, University of Pretoria. 

“Levera g ing Mob i l e T echnology f o r m-Lea r n ing: 3rd Generation Threaded Discussions” 

Timothy R. Hill, San Jose State University. 

“Accommodating Accounting Majors in a Required CIS Be ginning Programming Course. A p proach and 
Results” 
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David V. Beard, Kenneth J. Trimmer, Mark A. Bezik, Kenneth A. Smith, Idaho State University. 



Session 8 (12:30 -1:30) 

8 - Joint IAIM/ICIS Curriculum Presentation (Bayou 1) 

SESSION CHAIR: Raymond Papp, University of Tampa 

“IS 2002: An Update of the Information Systems Model Curriculum” 

Herbert E. Longenecker, Jr., University of South Alabama, David L. Feinstein, University of 
South Alabama, Gordon B. Davis, University of Minnesota, John T. Gorgone, Bentley College, 
and Joe Valacich, University of Washington. 



We hope you will join us for ICIER 2002 next December 13-15, 2002 in Barcelona, 

Spain 



O 
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ABSTRACT 

This paper starts from the premise that to be effective, e-education has to be intrinsically motivating. However, in 
contrast to much of the literature in the field, which focuses almost exclusively on the needs of students, the paper 
discusses THREE groups of stakeholders whose concerns and motivation have to be considered: students, instructors, 
and institutions. Following a critical review of the literature on e-education, which highlights some of the major 
themes that have attracted research so far, we proceed to introduce a model that integrates the needs of the above 
three stakeholders. The model is followed with a description of the Radical Model, an innovative approach to e- 
education that is an example of applying the proposed model in practice. We conclude with a discussion of the 
research implications from the model. 



INTRODUCTION 

In a recent editorial (Emurian, 200 1, pp 3-5), the author 
hails e-education as a revolution that would make the 
dream of “management of individual differences among 
learners” come true. In his editorial, the author lists a 
number of rhetorical questions that relate to the issues 
that he believes will be addressed by the advent of e- 
education, including: 

1. Where is it written that the pace of life must be 
controlled by an academic institution? 

2. Where is it written that a course grade must be 
frozen in time for ever? 

3. Where is it written that a student must be limited to 
a single evaluation occasion, without the opportunity 
for additional learning to achieve an intellectual 
criterion of excellence? 



4. Where is it written that the scale of an intellectual 
unit must be a traditional semester long course? 

The above questions reflect a series of issues thatare of 
importance to students and that e-education could 
address. Once these issues are addressed through the 
design of courses thatcan be started andfinished at any 
time and assessment procedures that allow students to 
repeat tasks indefinitely, the end result could, indeed, be 
a highly individualised learning experience forstudents. 
But is this what e-education is about, particularly in the 
context of universities? 

The underlying premise of this paper is that this is not 
the case. In order for e-education to succeed it has to 
cater to THREE stake holders and not just one. Creating 
an environment that is motivating to students is one of 
the major objectives of any educational technology. 
However, for such a technology to be sustained over 
time, it has to be intrinsically motivating to those who 
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manage it (instructors) and those who resource it 
(institutions). Followin g a critical rev iew of the literature 
on e-education, which highlights some of the themes that 
have been emphasised by previous research to date, we 
proceed to introduce amodel that integrates the needs of 
students, instructors and institutions. The model is 
followed with a case study that details the Radical 
Model, an innovative approach to e-education thatis an 
examp le of applyin g the mo del in practice. We conclude 
with a discussion of the research implications from the 
model. 

LITERATURE REVIEW 

The literature on e-education to dateseems to emphasise 
a number of themes. Following is a short reviewof these 
themes. 

First, there seems to be a debate over the TYPES of 
approaches to on-line teaching. One of the central 
models in this area, the Typology of Dispersion 
(Johansen, 1992), differentiates between on-line teaching 
that occurs at the same place and at the same time 
(Synchronous/Proximate), teaching that occurs at the 
same time but in different places (Anytime/Virtual), 
teaching that occurs at the same place but at different 
times (Synchronous/Dispersed), and teaching that occurs 
at different times and different places (Asynchronous/ 
Dispersed). Other writingsdiscuss specific technologies 
that can support the various teaching situations in the 
above model, such as presentationtechnologies (Leidner 
and Jarvenpaa, 1995) to support the same time/same 
place teaching, video conferencing to support same time/ 
different place teaching (Alavi, Wheeler, and Valacich, 
1995), Web page presentation, e-m ail and other Internet 
based technologies to support different time and 
different place teaching (Chizmar and Williams, 1996; 
Kuecheler, 1999). 

Second, there is a growing literature on underlying 
PHILOSOPHY of on-line teaching. One of the central 
models in this area, the Dimensions of Learning 
Theories approach, has been proposed by Leidner and 
Jarvenpaa (1995). The modeldifferentiatesbetween two 
broad philosophies of teaching. Objectivism, which 
holds that learning occurs in response to an external 
stimulus, and constructivism, which holds that 
knowledge is created in the mind of the learner. As a 
result, while the objectivism approach would lead to 
learning situations where knowledge is “delivered” to 



passive learners by an active instructor, the constructiv ist 
philosophy would result in learning situations where 
active learners create knowledge through interaction 
with each other. 

There is an emerging body of literature that looks at the 
implications of this model to on-line teaching (Passerini 
and Granger, 2000). The findings from this research 
seem to suggest that the objectivist approach does not 
result in significant benefits, namely, there are no 
significant differences between face to face and 
videoconference lectures (Alavi, Yoo and Vogel, 1997) 
and there are no significant differences between web site 
and audio supported learning and face to face learning 
(LaRose, Gregg and Eastin, 1998). However, the 
constructivist approach does seem to have relative 
benefits in that GSS supported classes seem to do better 
than face to face ones(Alavi, 1994), particularly in areas 
relating to critical thinking (Alavi, Wheeler and 
Valacich, 1995). Interestingly, while the quality of 
learning for the IT supp orted stude nts seems to be about 
the same as for the face-to-face ones, they appear to be 
less satisfied with the learning experience (Ocker and 
Yaverbaum, 1999). 

Finally, a third prominent theme in the literature on e- 
education is the discussion of its STRCTRAL 
antecedents. Here we find, on one hand the claim thate- 
education is a necessary evil imposed on universities 
because of declining resources and the necessity to 
reduce costs and expand m arkets (Ala vi, Yoo an d Voge 1, 
1997) and on the other the .fear that once universities 
embrace this innovation, it could result with a “second 
rate” education for students and a transformation of 
university instructors from creators of new knowledge 
(researchers) into assembly line labourers, delivering 
educational services to masses of virtual students (Klor 
de Alva, 2000). 

TOWARD AN INTEGRATIVE 
MODEL OF E-EDUCATION 

The above review suggests a need for integration of what 
appears to be several distinct bod ies of researc h. While 
the first body of research, on the technologies that 
support e-education, is important in terms of under- 
standing the tools that can be applied in this area, and 
while the second body of research, on the underlying 
philosophies of e-education, can help assess the 
effectiveness and efficiency of e-education in terms of 
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how it meets a given setof goals, both bodies of research 
are student centred in that they focus primarily on the 
needs of students. 

What is currently lacking in the literature is more 
emphasis on the needs of the two other stake-holders in 
the e-education game, namely, instructors and 
institutions. Figure 1 presents a pictorial depiction of the 
three dimensional integrative model that we are 
proposing as a basis for a future research agenda in this 
area. The three dimensions of the model reflect the 
motivational needs of students, instructors and 
institutions that are yet to be described. 

FIGURE 1 

INTEGRATIVE MODEL FOR E-EDUCATION 




Students 



The following are some issues that, based on current 
research, are likely to emerge as motivating factors for 
the three stake-holders and that could be the content of 
future versions of the proposed integrative model. 

Students 

Some of the issues that are likely to motivate students to 
engage in e-learning are the perception that this 
experience has resulted in the acquisition of relevant 
knowledge and skills, satisfaction with the interaction 
with the instructor and other students during the learning 
process, receipt of ample amounts of feedback on 
progress, and the receipt of an a fair grade at the end of 
the course. 



Instructors 

Some of the issues that are likely to motivate instructors 
to engage in e-learning are the perception that this 
experience has resulted in an enhancement of the 
instructors knowledge and skills, satisfaction with the 
interaction between the instructor and the students 
during the learning process, the perception of the effort 
invested in the teaching of course as reasonable, the 
perception that the instructor receives appropriate 
recognition and rewards for his/her investment in e- 
education. 

Institutions 

Some of the issues that are likely to motivate instructors 
to engage in e-learning are the perception that the 
institution is likely to gain from investing in this 
innovation, satisfaction with the process of changing the 
organization to one that engages in e-learning, once it 
has been undertaken, the perception that e-learning is 
cost-effective, the perception that the organization is 
likely to benefit from the investment in e-education in 
future. 

APPLYING THE 

INTEGRATIVE MODEL IN PRACTICE 

In the following sections we discuss an approach to e- 
education that is currently applied for a range of courses 
(Cook and Veach, 1997, Romm and Taylor, 2000, 
Roberts, Jones and Romm, 2000) atCentral Queensland 
University, Australia, including small post-graduate 
courses (with up to 20 students) and large under- 
graduate courses (with up to 100 students). The students 
are a combination of on-campus and distant learners, 
with both groups treated as one homogenous class. 

To date, this approach has been used to teach courses in 
Management of Information Systems and Electronic 
Commerce. Student responses to this approach have 
been very positive. One indication of this is that the two 
elective courses that which pioneered this approach on 
a large have gone up from zero to over 300 students in 
just about two years. 

The teaching materials for this approach (irrespective of 
what area is being taught) include: 

• A video which contains detailed explanations on how 
the course is run; 
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• A ten-page booklet “Course Outline” which describes 
all necessary information about the course (it is 
available on line as part of the course’s Web site and 
is provided to the students on a CD ROM and on hard 
copy); 

• A textbook; and 

• A class e-m ail list. 

The first thing that studentsdo once they read the course 
outline and watch the video is subscribe to the class e- 
mail list. They then introduce themselves to the class on- 
line so they can be divided into weekly presentation 
groups. The allocation to groups is completed by the 
second week of the semester. By this time, students are 
expected to establish contact with their virtual group 
members and start working ontheir assessment tasks. On 
week 3 of the semester, the first group makes its 
presentation to the class on-line. The presentation 
consists of an article (which the students have to enclose, 
attach, or simply establish a hyper-link to) and a critique 
that links the article with the reading in the book for the 
week. 

The presentation is made on Monday of each week. By 
Friday, each of the groups in the class is expected to 
comment on the presentation. On Sunday, the 
presentations for the week are read by the instructor 
along with the comments that were made by all the 
groups. All groups are marked every week for either 
their presentation or their comments about other 
students’ presentations. This procedure is repeated for 
ten weeks up until the end of the semester, with each 
week dedicated to an in-depth discussion of adifferent 
topic that is related to the reading for that week. 

The students’ mark for the course consists of 50% group 
mark for their performance in the group and 5 0% an 
individual exam. To make sure that students do not take 
advantage of their group membership, all groups are 
invitedto submit a consensus opinion of their members. 
Students are told right at the beginning of the semester 
that if the members of a particular group are in 
agreement that one member did NOT pull his or her 
weight, the mark of that student can be reduced by 10 
points. 

To conclude: 

The Radical Model makes efficientuse of the students’ 
interactions with each other. Even tho ugh, students have 

4 



some private interaction with the lecturer (“one to one”) 
and some interaction as a group, when the lecturer 
communicates with them on the class list (“one to 
many”), the bulk of their interactions in this approach is 
in the “many to many” mode, with the other students in 
their presentation groups and with the rest of th e students 
in the class through the class e-mail list. 

Throughout the semester, students are assessed on 1 1 
assessment tasks (including their group presentation, 
comm ents on other students’ presentations and an end of 
term exam). In a class of 1 00, they get 1 8 comments that 
represent the views of their own group members (nine 
members), as well as nine group comments representing 
the other 90 studentsin the class. Since this procedure is 
repeated every week, the students can receive over 100 
unit of inputs from their group members, the other 
groups, and from the lecturerby the end of the semester. 
Note, that most of the feedback on one’s performance 
comes from the OTHER students - not the lecturer. 

It should be noted that even though class interaction is 
the means through which teaching takes place, the 
Radical Model does not result in the list being flooded 
by e-mail messages. As indicated in the previous 
sections, students are instructed to refrain fromusingthe 
class list for unlimited expression. The place for such 
interaction is supposed to be the small presentation 
groups that they establish to support their group work. 
The messages that end up being posted on the class list 
are messages from the list moderator (the lecturer) 
“formal” presentations of the students’ work, and 
comm ents by the other groups aboutthese presentations. 

The Radical Model helps develop students’ communi- 
cation and other “soft skills.” In addition to learning 
about the content area for the semester, students learn 
important on-line skills such as how to set up their e- 
mail lists, how to be citizens of an on-line community, 
and how to contribute to a virtual team, including 
dividingthe work between the team members, resolving 
conflicts, developing ideas and projects, and providing 
positive feedback to others about their work. 

Through the involvement of students from diverse 
backgrounds (many of whom are fully employed) 
students learn about how organizations use the abstract 
concepts that are mentioned in the readings. They also 
learn about relevant legislation and ethical issues. 

The Radical Model is “flexible” for both the instructor 
and the students. This approach increases flexibility for 
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students, because the students don’t have to subm it hard 
copy assignments (hence, nothing can get lost through 
the system). They get to know if their submission was 
successful immediately when they see it posted on the 
class list. As well as this, if something happens to 
preclude an individual student’s contribution during the 
semester, time out and compensation work can be 
negotiated within groups. In fact, students don’t need to 
ever negotiate with the lecturer on late submission, 
special consideration, etc. All negotiations on these 
issues are carried out within the group. 

Students have further flexibility in not having to down- 
load large am ounts of data from the class Web site (there 
is nothing onthe web site other than the Course Outline). 
They don’t need to buy any books other than the course 
textbook, and even this book can be shared between 
them up until the end ofthe semester, as all assessment 
tasks are group based. Because all learning is facilitated 
by the class list, the students can engage in class 
activities from home, work, or while travelling. Further 
flexibility to the students is provided through the 
students’ selection of supplementary readings for class 
discussion by themselves. As a result, students get to 
read quite a large number of articles on topical issues 
that are of interest to them rather than forced to read 
articles selected by the instructor. 

Lecturer flexibility is also an enormous advantage of the 
radical model. Since the package for this course does not 
include a Study Guide, there is no need to update one 
every semeste r. Since the co urse is in no way dependent 
on a textbook, there is no need to m odify or change it in 
any way if and when there is a need to change a 
textbook. In fact, preparing study materials for a new 
semester should not take more than a few minutes, given 
that nothing substantial has to change. 

As for on-going teaching; reading the weekly 
presentation and the comments by the other groups 
(students are restricted to two pages or two screens 
maximum per critique or comment on other people’s 
critique), takes two to three hours per week. This can be 
done from anywhere, including from home or from a 
conference. Theoretically, even if the lecturer is totally 
incapacitated, another person can easily take over and do 
the on-going weekly assessment, without inconveni- 
encing the students. 

Note that this design is also advantageous from a legal 
perspective. Since articles by other authors are not used 



as part of the course Web site, there is no infringement 
on other people’s copy-rights. 

The most important aspect about this model is that no 
matter how many students are in the class, the amount of 
work for the lecturer is the same!! No matter how many 
students are in the class, 10 or 100, the lecturer ends up 
checking 1 0 presentations of one page each per week for 
ten weeks. If the class consists of 10 students, these 10 
pages of text represent the work of each of them. If the 
class consists of 100 students, the ten pages will 
represent the work of the ten groups into which the 
students have been divided. Thus, the amount of 
semester marking for the lecturer remains the same, 
irrespective of the number of students in the class. 

WHY IS THE RADICAL 
APPROACH AN APPLICATION 
OF THE INTEGRATIVE MODEL? 

It is our belief that the Radical M odel works because it 
represents an integration of the three components ofthe 
Integrative E-education model. To demonstrate this 
point, we would like to go back to the issues that were 
mentioned previously as contributing to the motivation 
of the three stake-holders to engage in e-education. 

Students 

The Radical approach ismotivating to students because 
in addition to acquisition of relevant knowledge and 
skills, they also receive large amount of feedback from 
the instructor and their fellow students. Because of its 
“constructivist” philosophy, the model is also associated 
with ample opportunities for interaction between the 
students and the instructor and among the students. Since 
50% of the mark in this course is based on an individual 
exam, the students feel that their efforts both as 
individuals and as a group are acknowledged and fairly 
rewarded. 

Instructors 

Instructors are motivated to use this approach because by 
allowing the students to “create” the course (through 
selection of the weekly readings and leadership of the 
class discussion), there is an opportunity for instructors 
to expand their knowledge and skills as a result of 
teaching the course. Since most of the administrative 
issues that are associated with the teaching of the course 
(handling late submissions, appeals, etc) are resolved 
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WITHIN the groups without any input from the 
instructor, the overall experience of interacting with the 
class is exceptionally positive for the instructor. Since 
students are basically teaching each other, theeffort that 
is involved in teaching the class is minimal, hence 
contributing the perception of instructors that they are 
not investing in the virtual class more time and effort 
than they would in a face to face class. 

Institutions 

The above case did not elaborate on the organisational 
context of the Radical Approach. However, from the list 
of tools that are used to supportthis approach, it is clear 
that this approach involves minimal investment on the 
part of the institution (the only requirement is to 
establish an e-mail list and have the students subscribe to 
the list). At least from this perspec ti ve, this approach can 
be seen as highly cost-effective for institutions, and, as 
such, highly motivating. 

RESEARCH AGENDA EMANATING 
FROM THE INTEGRATIVE MODEL 

The underlying premise of this paper, that the success of 
e-learning should be assessed in terms of its motivating 
potential to students, instructors, and institutions could 
be researched in the following ways. 

1. Outcomes. Future research could compare different 
on-line teaching styles in terms of their effect on 
outcome variables such students, instructors and 
institutional satisfaction, quality of the learning 
process, etc. Once undertaken, such research could 
determine empirically the dynamics between the 
three stake-holders that produces successful e- 
learning. 

2. Process. An analysis of the interactions in the on- 
line class and in organizations that use e-education 
on a large scale, particularly from a qualitative 
longitudinal perspective, can reveal patterns of 
communication and group dynamics thatare typical 
of effective versus ineffective e-education 
environments. 

3. Antecedents. The effect of a range of moderating 
variables on both the outcome and the process of 
effective e-education can be explored. Mediating 



variables could include: demographic variables (gender, 
age, socio-economic class, ethnicity), attitudinal 
variables (learning style, preference to work in the 
distant mode), institutional variables (course, program 
studied)and global variables (national culture). All these 
variablesshould, of course, be explored in termsof their 
effect on the perceptions of members of all three stake- 
holder groups. 
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ABSTRACT 

This paper proposes and empirically tests a specific knowledge management technology aimed at stimulating creative 
and innovative performance of students in software development courses. The technology is based on a solo 
brainstorming method that provides students with external stimuli and exposes them to a large number of ideas over 
a short period of time. An empirical test was conducted using 16 volunteer student subjects. It reveals a highly 
beneficial effect of the technology on the participants ' ability to specify requirements for a new software product. In 
particular, interaction with the technology resulted in a significant increase in the total number of ideas generated 
by the participants, as well as the number of relevant categories of issues addressed. These results suggest that the 
technology may be useful in facilitating learning and perfofwance in informatics subjects involving creative thinking 
and problem solving. 



INTRODUCTION 

There is a widespread recognition in the business 
literature that creativity and innovation are the major 
sources of the economic growth and competitive 
advantage of today’s organisations (Drucker, 1985; 
Satzinger et al., 1999; Tomas, 1999). It is therefore not 
surprising that surveys show that these two issues are 
among the top priorities for senior executives in industry 
today (BW, 1998). There is also an acknowledgement 
that software has emerged as central to all sophisticated 
innovations(Quinnet al., 1997). Inmany cases, software 
is the end product itself, or it is the highest value 
component in the end product. This is true for most of 
the fastest growing industries including computer, 
entertainment, communications, advertising, logistics 
and financial services. In other cases, software facilitates 
most stages of the value creation and innovation 
processes. 



The changing economic landscape, particularly the 
growing importance of software-based innovations, 
suggests the need for better management of professional 
knowledge of future software developers. This, in turn, 
requires an appropriate response from education and 
training institutions. The purpose of this study is to 
address the issue of creativity and innovation by 
describing and testing a specific educational technology 
aimed at stimulating creative thinking and idea 
generation of students in software development courses. 

Creativity can be defined as the production of novel and 
appropriate ideas, solutions and work processes (Shalley 
and Perry-Smith, 2001). While newness and novelty are 
the key dimensions of creative expressions, 
appropriateness is also an essential requirement in the 
context of problem solving and innovation. Generally, 
the education sector should nurture creativity, so that 
students can be successful in their future roles as 
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innovative professionals and business people. More 
specifically, it is of utmost importance that informatics 
students be given an opportunity to develop and apply 
creative and innovative skills to software processes and 
products. 

Some theorists belie ve that creativity isreserved only for 
the gifted, while others see it as a skill that can be 
learned (Ford, 1996; Marakas, 1998). We see creativity 
as a property of a thought process that can be acquired 
through instruction and practice. Therefore, the main 
objective of this study is to: (i) describe a specific 
technology based on a brainstorming method as a 
potentially suitable tool for stimulating informatics 
students’ creative thinking and idea generation, and (ii) 
test empirically the effectiveness of the technology in 
enhancing students’ creative performance in the context 
of a requirements specification for a new software 
product, in particular an information system. 

TECHNOLOGY DESCRIPTION 

There are various types of knowledge managem ent tools 
available for supporting different stages of the 
knowledge management process. According to Wensley 
(2000), the majority of these tools are useful in two 
ways: they may assist in makingtacit knowledge explicit 
and they may be used to encode explicit knowledge. The 
technology in this study is fundam entally different, as it 
aims at supporting knowledge creation rather than 
facilitation of knowledge transfer through extemalisation 
and codification. 

The technology is basedon a solo brainstorming method. 
This is an individual brainstorming technique in which 
the participant interacts with a set of documents and 
identifiesissues from these. Typically, the brainstorming 
session includes reading and editing of stimulating 
material, typing a summary of the documents, making 
lateral comments and nominating issuesfrom them to be 
followed up (Aurum, 1997, 1999). The protocol 
encourages lateral thinking and brings a formal setting to 
the process of issue identification. It also incorporates 
Osborn's (1957) four basic brainstorming rules: no 
criticism is allowed while interacting with documents, 
freewheeling is welcome, quantity is wanted, and 
combination and improvement are sought. Earlier, the 
method has been successfully applied in a number of 
decision making tasks (Aurum, 1997; Aurum and 
Martin, 1999). This suggests that the technology based 
on solo brainstorming may be useful in stimulating the 
creative thinking of students in software development 



tasks, particularly in the requirements specification 
phase. 

Essentially, the technology provides support for the 
following activities: 

(i) Documenting the requirements that students 
considered relevant to the given software 
development task. These were the issues that they 
were aware of before interacting with the stimulating 
material; 

(ii) Interacting with a series ofdocuments one at a time, 
and noting issues as they arose or new ideas as they 
were generated. The intention was to capture users’ 
reasoning, opinions, knowledge andlateral thoughts 
which went into their reaction to the stimuli. In this 
way, users could more confidently approach the task 
of identifyin g issues and generating ideas; 

(iii) Documenting ideas that occurred to students after 
interacting with the documents. T he objective was to 
enable users to express their ideas, comments and 
knowledge, combined with the data that they 
interacted with. Since users were presented with a 
copy of their comments from the interaction, they 
could go through the comments and identify the 
relevant issues. 

The technology also incorporates a set of stimulating 
documents as primary external information sources 
regarding cultural heritage of Australia. These included 
abstracts of journal and conference papers which were 
retrieved from bibliographic databases. It was expected 
that the subjects would explore the given problem from 
many different angles by restructuring and rearranging 
information available in the abstracts and combining 
them with their expertise. It was also important not to 
prejudge while interacting with abstracts. The critical 
aspect was to prevent brainstorming from a 
prejudgement of an issue set that was relevant. 
Therefore, sometimes abstracts addressed general issues, 
in which case the specifics of the local case (Australia) 
had to be considered by the participants. Sometimes, 
experiences from Venice in Italy or from Louvre 
museum in Paris, France were not expected to be 
relevant to Australian situation at all. In any case, the 
role of these documents was to trigger ideas. 

It was also expected that in this interactive environment 
the notions given in abstracts would prompt recognition 
and processing by individuals. Previous experience and 
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expertise were also consideredimportant. The responses 
could come from direct reactions as well as stimulated 
recollections. These were both potentially valuable. The 
intention was to tease out such reactions to the full. 
These principles are assumed to be valuable for any 
situation in which creative and innovative ideas are 
desirable. In order to maintain this atmosphere, during 
the interacti on stage, the subjects were required to follow 
the brainstorming protocol. 

In developing the technology, a particularattention was 
also paid to designing the interface. It was important to 
prevent substantial cognitive resources being diverted 
from the task in response to demands from the user- 
interface. The aim was to produce an interface that 
would have minimal impact on cognitive load, one 
which could be learned easily by a novice user and yet 
was comprehensive enough to satisfy the experienced 
user(Aurum et al., 1995). 

EMPIRICAL TEST 

The technology was empirically tested in a simulated 
business situation based on Collins et al. (1996). It 
involved a fictitious organisation, The Cultural Heritage 
Authority, whose job was to coordinate the marketing of 
cultural heritage in Australia. Subjects adopted the role 
of user/developer working for the authority. Their task 
was to generate ideas with respect to the anticipated 
requirements for the authority’s new software-based 
marketing system. During the task performance, the 
subjects were required to produce two sets of documents: 
one before and one after interacting with the technology 
provided to aid their task. Before interaction, subjects 
were asked to identify and record a range of potential 
issues they believed were important for marketing of 
cultural heritage in Australia based solely on their past 
knowledge and experience. Then, they were asked to 
interact with the technology and edit and read all 
available material. After reading and editing each 
abstract, the subjects were required to produce a second 
written document, identifying additional issues and 
adding commentary notes. 

A total of 16 subjects participated in the study on a 
voluntary basis. The participants were drawn from the 
pool of graduate students attending courses in software 
development process at the large Australian university. 
Each received a monetary incentive of $45 for their 
participation. The experimental session was conducted 
in amicrocomputer laboratory. On arrival, subjects were 



seated at individual workstations and worked alone. 
They received instructions regarding the case study and 
task requirements. They also had an opportunity for 
practice prior to commencing the experiment and for 
asking questions during the experiment. The session 
lasted 3 hours. 

Subjects’ accomplishment in the task was evaluated in 
terms of the total number of relevant ideas generated 
(denoted total ideas), as well as the number of relevant 
categories of issues addressed (out of 19) as assessedby 
an expert judge (denoted total categories ). The judge 
was a software developer with over 30 years of 
experience. The judge examined students’ ideas for 
relevance and assigned them to appropriate categories 
and sub-categories. The classification scheme enabled 
examination ofthe changes in the quantity, as well as the 
quality of ideas generated by the students due to 
technology use. 

RESULTS 

In order to understand the effects of the technology on 
individual participants’ creative performance we 
analysed statistically the changes in the number and 
nature of ideas generated ‘after’ the interaction with the 
technology compared to those ‘before .’ The paired T-test 
was selected asthe most suitablemethod forthe analysis 
(Huck et al., 1974). Re suits of the analysis are shown in 
Table 1. 

Overall, the results of the analysis performed indicate a 
significantpositive impact ofthe technology on both the 
quantitative and qualitative aspects of the participants 5 
creative performance. Table 1 shows significanteffects 
on both dependent variables of performance, total ideas 
and total catego ries. 

With respect to the quantitative aspect of perfomiance, 
the results indicate a highly significant positive impact 
of technology on total number of relevant ideas 
generatedby the participants. As shown in Table 1, there 
was a significantincrease in the number of relevant ideas 
generated due to using the technology. More 
specifically, the mean value for total ideas increased by 
9.50, from 7.69 before to 1 7.19 after the interaction (p < 
0 . 001 ). 

More interestingly, the results indicate a highly 
significant positive impact of technology on the 
qualitative aspect of participants ’ performance. Table 1 



10 



Proceedings of the 16 h Annual Conference of the International Academy of Information Management 



TABLE 1 

RESULTS OF PAIRED T-TEST ANALYSES ON 
DEPENDANT VARIABLES OF PARTICIPANTS’ PERFORMANCE 



Dependent 
Variable Tested 


Unit of 
Measurement 


Mean Value 
Before 
Interaction 


Mean Value 
After 

Interaction 


Direction and 
Amount of 
Change 


Significance of 
Change 
(p value) 


Total Ideas 


Number 


7.69 


17.19 


4*9.50 


0.000* 


Total Categories 


Number 


4.31 


8.25 


4*3.94 


0.000* 




(%) 


(23) 


(43) 


(+20) 





*p < 0.001 

showsthat there was a significantincrease in the number 
of relevant categories addressed by the participants as a 
result of their use of the technology. The mean number 
of different categories addressed expanded on average 
by 3.94 (or 20 %), from 4.31 (or 23 %) before to 8.25 (or 
43%) after the interaction with the technology 

(p <0.001). 

DISCUSSION 

The experimental results clearly indicate that a solo- 
brainstormingbased technology had apowerfiil effect on 
students’ creative performance in the software require- 
ments specification task. The effect was evident in both 
the quantitative and qualitative aspects of the 
performance evaluated. Users were found to generate 
significantly greater number of relevant ideas, as well as 
address a significantly greater number of relevant 
categories of issues as the resultof their interaction with 
the technology. 

The results of this study provide substantial support for 
the view that creative performance can be enhanced by 
appropriate instruction, as suggested by some theorists 
(Ford, 1996; Marakas, 1998). The instructional tech- 
nology studied here was found to contribute to a 
significantimprovement in both the quantity and quality 
of ideas generated by the participating students. These 
results also agree with our earlier findings from an 
empirical study conducted in the decision making 
context (Aurum et al., 2001). Essentially, the results 
support the idea that thinking applications can be 
developed, learnt, practiced and used to generate ideas. 
Thus, they can enable an individual to think, provided 
that the principles are clearly understood. 



It has also been suggested in the literature (Satzingeret 
al., 1 999) that the idea generation method is one of the 
most important sources of encouraging creativity. The 
results of our study indicate that the brainstorming 
technique underlying our instructional technology was a 
highly appropriate method for stimulating creative 
thinking and idea generation in a software development 
task. Students were found to generate a significantly 
greater number and variety of relevant system 
requirements ideas afterthan before asolo-brainstorming 
session. Essentially, the session helped students uncover 
ideas without being constrained, stimulate their own 
thinking by external influences and capture all of their 
thoughts. 

The findings of the study also support the proposition 
that an electronic tool following a specific creativity 
technique can assist the creative process (Sridhar, 2001). 
One of the main advantages ofsuch a tool is the speed at 
which ideas can be produced. Furthennore, the ideas can 
be stored and revisited at the later time. Thetool canalso 
provide a variety of stimuli that enhance imagination. 
The electronic technology implemented in this study 
provided all of the above, plus a written protocol that 
broughta formal structure to the ideageneration process. 

Our findings may have some important implications for 
the education sector. Generally, they suggest that the 
education sector first needs to acknowledge the 
importance of teaching students how to think creatively. 
Then, it needs to implement courses whose purpose is to 
prepare students for the knowledge based economy. It 
appears that some governments and institutions are 
starting to emphasise the significance of promoting 
creative thinking of young through education 
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(Sunderland, 2000) and are beginning to implement 
changes in courses taught in business schools (Sangran, 
2001). The results of this study suggest that the 
instructional technology tested here may be a useful 
teaching tool in a variety of courses involving creative 
thinking and problem solving. They also suggest that it 
is likely to be most valuable in situations where the 
problem is unstructured, goals indistinct, and the 
outcome ofan action cannot always be clearly identified. 
The tool is a rather generic one, since it uses a technique 
that can be applied to a variety of scenarios and can help 
people process relevant documents whilst identifying 
issues. These documents act like a ‘trigger’ to stimulate 
domain specific ideas from users. 

While the current study provides a number of interesting 
findings, some caution is necessary regarding their 
generalisability due to a number of limiting factors. The 
application of laboratory conditions is a limitation of this 
study. We believe that in field work, the users’ 
individual achievements would improve even further. 
We also speculate that the performance of users in an 
interaction session can be affected by their state of mind 
or previous experience. In some cases, the user may 
already be ‘aware’ of the requirements and may have the 
desire to look for some particular aspects rather than 
identifying issues in general. There may also be other 
situations where the user may only want to see what the 
literature is about but not be interested in identifying any 
requirements. In such cases, the user will act differently 
in the course of interaction with the abstracts. The 
emphasis of the present study was on individual 
students. It would be interesting to examine the effect of 
the technology on creative performance of groups. 
Future research may address some of these issues. 

CONCLUSION 

This study proposed and empirically tested a specific 
technology for stimulating creative problem solving of 
software developers. The technology was designed on 
the basis of a solo brainstorming technique. The essence 
of tool was to provide users with external stimuli and 
expose them to a large number of ideas over a short 
period of time. The tool was tested in the context of 
software requirements specification task. The results of 
the test indicated that the tool was quite helpful. Users 
were able to improve their creative performance and 
generate greater number of relevant ideas and address a 
more relevant categories of requirements. Future 
research is recommended to address current study 
limitations. 
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ABSTRACT 

Distance learning is quickly becoming an accepted and even necessary part of college and university programs. As 
more colleges and universities join the growing ranks of institutions offering distance learning , educators and 
administrators are struggling with the issue of how to assess student success in this new and largely untested 
environment. Many distance learning providers, and even some institutions, have developed short surveys that are 
designed to gauge whether a student is prepared to undertake distance learning. These tests, however, may not 
accurately assess a student’s predisposition and learning style. This paper highlights and compares the use of 
different learning style inventories as a means to formally and empirically assess learning styles. Students in both 
distance learning and traditional classroom courses were given several of these inventories and their progress was 
tracked. Initial results indicate that some of these can be used as a successful predictor of student performance and 
may be useful for students and administrators in determining whether or not the student should undertake a distance 
learning course or program. The paper concludes with some suggestions and implications for educators on distance 
learning. 



INTRODUCTION 

Distance learning has become an established, accepted 
part of many college and university offerings and, in 
some cases, represents a major portion of the curricula. 
Given the technological advances and broad reach of the 
Internet, this is not surprising. Students and admini- 
strators alike are demanding such programs given the 
changing student demographics and societal needs. As 
the traditional college populations change, the need for 
programs which address convenient, flexible, and 
adaptable learning increases. Life-long learning and re- 
tooling also necessitate programs which can 
accommodate full-time workers and those with familial 
obligations. Need notwithstanding, distance learning 
providers have not spent much time investigating 
whether or not this new pedagogy is suitable for 
everyone and how to assess who might benefit from it. 

Many institutions offer distance-learning courses and/or 
programs but have little or no pre-assessment for 
students. They feel that the need for such courses and 
programs is justified, however they often do not provide 



a way for prospective students to assess their level of 
readiness for such programs. This is especially true for 
those students who have been away from the classroom 
environment for some time and are now returning to 
continue a course of study or re-tool in a new area. This 
paper begins with an exploration of some of the ways in 
which students are currently tested, highlights and 
compares several different learning style inventories, and 
presents initial findings from both on-campus and on- 
line classes with respect to student learning styles. It 
concludes with general suggestions for educators and 
administrators on how to assess student readiness for this 
new environment. Future research will gather data from 
a much larger population to assess which learning style 
assessment, if any, is most suitable for assessing student 
readiness for this new and largely unexplored pedagogy. 

EXISTING ASSESSMENTS 

A few of the commercial distance learning providers 
have developed their own self-assessment surveys or 
questionnaires to assist students in deciding whether or 
not to undertake a distance learning course or program. 
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Some are very short, others are more comprehensive and 
thorough. For example, one of the providers, 
eCollege™, uses a short 10-question quiz to assess 



potential student success (http://www.onlinecsu.ctstateu. 
edu/index.real?action=IsOnline). 



» Sludent/FacuUy Login 



• About OniineCSU 
o Course Catalog 
<h Registration 

» Financial Information 
« Degree Information 
» Accreditation 
e Academic and 
Student Services 



» Student FAQs 
o Faculty/staff FAQs 

« Technical Information 
» Browser Test 
« Help Desk 
a Inquiry 
« Site Map 



Link to All Four 
CSU Universities 



Is Online Learning For Me? 

Take the Quiz ... 

1. My need to take this course is: 

O high- I need it immediately for a degree, job, or other important reason. 
(•' moderate- I could take it on campus later or substitute another course. 
G low- it is a personal interest that could be postponed. 

2. Having face-to-face interaction is: 

O not particularly important to me. 

<? somewhat important to me. 

C very important to me. 

3. I would classify myself as someone who: 

C often gets things done ahead of time. 

<?? needs reminding to get things done on time. 

O puts things off until the last minute. 

4. Classroom discussion is: 

O rarely helpful to me. 

<5* sometimes helpful to me. 

C almost always helpful to me. 



5. When an instructor hands out directions for an assignment, I prefer: 

O figuring out the instructions myself. 

trying to follow the directions on my own, then asking for help as needed. 

O having the instructions explained to me. 

6. I need faculty to constantly remind me of due dates and assignments: 

O rarely. 

sometimes. 

O often. 

7. Considering my professional and personal schedule, the amount of time I have 
to work on an online course is: 

O more than for a campus course. 

G the same as for a class on campus. 

O less than for a class on campus. 

8. When I am asked to use email, computers, or other new technologies 
presented to me: 

O I look forward to learning new skills. 

<5 I feel apprehensive, but try anyway. 

O I put it off or try to avoid it. 

9. As a reader, I would classify myself as: 

C good- I usually understand the text without help, 

<*> average- I sometimes need help to understand the text. 

C below average- I often need help to understand the text. 



10. If 1 have to go to campus to take exams or complete work: 

G I have difficulty getting to campus, even in the evenings and on weekends. 
p I may miss some lab assignments or exam deadlines if campus labs are not 
open evenings and weekends. 

Ci I can go to campus anytime. 
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The results of the exam, given a “middle of the road” set of responses as indicated above, are: 



ft 



m 



Cofrp(*k«s5taw 

$suau 



* Welcome 

* Student/Faculty Login 

* About OnllneCSU 

* Course Catalog 

* Registration 

* Financial Information 

* Degree Information 

* Accreditation 

* Academic and 
Student Services 

* Student FAQs 

* Faculty/Staff FAQs 

* Technical Information 

* Browser Test 

* Help Desk 

* Inquiry 

* Site Map 



Link to All Four 
CSU Universities 






CENTRAL 

casKEcrtctrr 







Western 
i Co nnecticut 
j State 

/ University 



Results 



You scored: 20 



20 points or higher- an online course is a real possibility for you. 

Between 11 and 20 points- an online course may work for you, but you may 
need to make a few adjustments in your schedule and study habits to succeed. 
Less than 10 points- an online course may not currently be the best alternative 
for you; talk to your counselor. 

No matter what you scored, remember that online learning is not easy. Your 
professor will demand at least the same quality of work as they would receive 
from you in a face-to-face classroom. A similar number of hours will need to be 
committed throughout the course of a semester for an online course as to a face- 
to-face course. Remember that your course may include deadlines and 
instructions on assignments, but there will not be anyone telling you to "turn in 
your assignment." Online learning is convenient- you do not have to commute to 
campus, and you can attend class at your convenience- either early in the 
morning, late at night, or anytime in-between. Just remember, no matter when 
you decide to study, your professor will hold you accountable- so study hard, and 
have a great terml 



eCollege' 






§g 




(http://www.onlinecsu.ctstateu.edu/index.real?action=IsOnline) 



ANALYSIS 

While the results may be helpful to a student, the point 
system illustrated above is vague and a variation on only 
one or two answers could result in a recommendation 
against distance learning. While the eCollege™ survey 
is easy to take, the accuracy of the results might be 
questionable in that students are not asked specific 
questions about their learning styles but rather more 
about the environment (e.g., their level of comfort with 
technology, ability to visit campus, and/or need for 
contact with the instructor). 

It is interesting to note that as of mid-2001, national 
institutions such as the University of Phoenix™, Jones 
International University™ and Western Governors 
University™ do not offer any type of pre-assessment. 
They encourage the student to enroll and work with a 



counselor and/or technical support personnel to bring 
them up to speed in their environment. This is a 
common approach, but one that may subject the student 
to unnecessary stress as they attempt to work out their 
difficulties and determine the suitability of a distributed 
learning environment. 

So how does a student determine if he/she will be 
successful in a distance-learning environment? They 
could enroll in one of these national or local institutions 
and “learn the ropes” and hope that they will be able to 
handle the rigors of learning without the benefit of an 
actual classroom setting. For older students and those 
who already have substantial work experience and/or a 
previous degree, this may not be a difficult adaptation. 
Students who will potentially require more assistance are 
those whose learning styles may not be amenable to a 
non-classroom setting. Local and regional institutions in 



16 



Proceedings of the 16 th Annual Conference of the International Academy for Information Management 




BEST COPY AVAILABLE 



28 










particular may target their own students or those in the 
local area rather than a national or international clientele. 
It is for these institutions that a more comprehensive 
assessment of learning style is beneficial (Sternberg, & 
Grigorenko, 1997). 

OVERVIEW OF LEARNING INVENTORIES 

There are numerous learning style assessments currently 
available. Some were designed for learning assessment 
in general while others have been adapted or modified to 
accommodate many of the newer learning styles and 
pedagogies. 

One of the oldest and most well known ways of 
assigning students to a given learning style is that of 
Kolb’s (1984) Learning Style Inventory (LSI). This 
assessment inventory consists of 36 words in 9 
groupings of 4 each. The student is asked to rank each 
of the sets of words on a 1 to 4 scale, with 1 equating to 
least like the person, 4 being the most like the person. 
The four columns of words correspond to four learning 
style scales: concrete experience (CE), reflective 

observation (RO), abstract conceptualization (AC), and 
active experimentation (AE). Kolb uses Jung’s (1977) 
typologies as the main foundation in the development of 
these learning styles. For example, the abstract 
conceptualization (AC) style “focuses on using logic, 
ideas, and concepts. It emphasizes thinking as opposed 
to feeling” while the concrete experimentation (CE) style 
“focuses on being involved in experiences and dealing 
with human situations in a personal way. It emphasizes 
feeling as opposed to thinking” (Kolb, 1984, 68-69). 

Kolb’s LSI has been criticized for its low reliability and 
validity measures (Freeman & Stumpf, 1978; Holman, 
Pavlice & Thorpe, 1997; Lamb & Certo, 1978; West, 
1982), yet it has received equal support as a way of 
illustrating the different approaches to learning (Abbey 
et al, 1985; Kruzich et al, 1986; Nulty & Barrett, 1996; 
Raschick et al, 1998). As a result of the diverse opinions 
and findings, other methodologies are needed to provide 
a more stable platform with respect to validity and 
reliability of such an assessment. Other methodologies 
which will be included in this investigation include Tait 
and Entwistle’s ASSIST (1996), Solomon and Felder’s 
(1996) Index of Learning Styles (ILS), Honey and 
Mumford’s (1992) Learning Styles Questionnaire (LSQ) 
and the Academic Self-Efficacy Scale (Eachus, 1993). 

The ASSIST (Approaches and Study Skills Inventory for 
Students) instrument developed by Tait and Entwistle 



(1996) is a 38-item inventory which attempts to identify 
students with weak study strategies. It has four 
sub scales which measure four approaches of studying 
and academic aptitude. The scales are deep (intention to 
understand, relation of ideas, active learning), surface 
(intention to reproduce, unrelated memorizing, passive 
learning), strategic (study organization, time 
management, intention to excel), and apathetic (lack of 
direction and interest). Students respond to items 
relating to each of these approaches along a five-point 
likert scale from “agree” to “disagree”. A score for each 
of the approaches is determined by summing the scores 
from each of the items corresponding to each subscale. 

Solomon and Felder’s (1996) Index of Learning Styles 
(ILS), originally developed for engineering students, 
focuses on four bi-polar preference for learning scales. 
These include Active-Reflective, Sensing-Intuitive, 
Visual-Verbal, and Sequential-Global (Felder & 
Silverman, 1988). Active learners are those who leam 
by trying things and working with others. Reflective 
learners prefer to think things through and work alone. 
Sensing learners are oriented toward facts and 
procedures while Intuitive learners are more conceptual, 
innovative and focus on theories and meanings. Visual 
learners prefer visual representations of material such as 
pictures, diagrams and charts while verbal learners prefer 
written or spoken explanations. Sequential learners are 
linear and orderly in their thinking and leam in small 
incremental steps while Global learners are holistic 
thinkers who leam in large leaps. These bi-polar scales 
offer a good basis for comparison of learning types. 

The Learning Styles Questionnaire (LSQ) developed by 
Honey and Mumford (1992) identifies four types of 
learners, Activists (e.g. enjoy new experiences, make 
intuitive decisions, dislike structure), Theorists (e.g. 
focus on ideas, logic and systematic planning, mistrust 
intuition), Pragmatists (e.g. favor practical approaches, 
group work, debate, risk-taking), and Reflectors (e.g. 
observe and describe, try to predict outcomes, try to 
understand meaning). According to the authors, 
individuals tend to rely on one of these approaches when 
they are engaged in learning. 

Finally, the Academic Self-Efficacy Scale (Eachus, 
1993) is a 23-item scale which assesses the extent to 
which students believe they have the ability to exert 
control over their academic environment. By totaling 
the scores from the items, a self-efficacy score can be 
determined. This can be useful for students to determine 
the extent to which a distance-learning environment will 
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be suitable for them. By knowing their level of self- 
efficacy, they will be in a better position to make a 
decision as to whether or not to pursue a more traditional 
course of study. 

METHODOLOGY & PRELIMINARY RESULTS 

Each of the assessment inventories highlighted above 
were administered to students in an on-campus 
sophomore level computer class focusing on hardware, 
software and networking. The same inventories were 
administered to students in a distance-learning course 
conducted during the summer of 2001. The course 
covered the same content as the on-campus course, but 
was delivered entirely over the Internet. 

Preliminary analysis regarding each of the learning 
styles is presented below. A total of 40 students took 
part in this pilot study. 25 took the traditional on- 
campus class and 15 took the distance-learning class 
over the summer. The mean age was 19 for the on- 
campus class and 20 for the distance-learning class. 
There were 15 women and 10 men in the on-campus 
section and 1 1 men and 4 women in the on-line section. 
All were classified as full-time students although 80% 
reported working at least 20 hours per week while they 
took the courses. It must be noted here that this small 
sample size prevents the results that follow from being 
generalized to the general population. It is the author’s 
intention to undertake a full study given the promising 
results of this initial pilot study. 

Kolb categorizes four learning styles: Concrete 

Experience (CE), Reflective Observation (RO), Abstract 
Conceptualization (AC), and Active Experimentation 
(AE). Analysis of the sample indicates that 75% of the 
students responded as Abstract Conceptualizes (AC) 
and 85% classified themselves as Active Experimenters 
(AE). Given the subject of the course, computer 
hardware and software, this is not surprising. It is 
interesting to note that more women (13%) then men 
(7%) categorized themselves as Reflective Observers 
(RO). This might be indicative of male-female 
personality differences, but this cannot be verified given 
the sample. 

Analysis of Tait and Entwistle’s (1996) ASSIST 
(Approaches and Study Skills Inventory for Students) 
instrument indicates that men (12%) are more inclined to 
be surface learners (e.g. more passive and inclined to 
memorize) than women (8%). Women outnumber men 



almost two-to-one in terms of deep (active learning, idea 
relation) learning (28% to 15%). Few students indicated 
an apathetic learning style and most were highly focused 
on strategic learning. Incidentally, this focus was much 
more pronounced in the on-line class (82%) than in the 
on-campus class (68%), possibly suggesting a greater 
need for time management and organization. This may 
have been the result, however of the shortened (six- 
week) class time period. 

Solomon and Felder’s (1996) Index of Learning Styles 
(ILS) focuses on four bi-polar scales. Active Learners 
try things out and prefer working with others while 
Reflective learners prefer to work alone and think things 
through. Students in both classes were largely 
Reflective learners, however this may be the result of 
their age and lack of experience with group settings. 
Sensing learners are more fact-oriented and Intuitive 
learners are more conceptual. Not surprisingly, students 
were more Sensing (87%) than Intuitive (56%), probably 
a result of their chosen major, computer information 
systems. People with an aptitude toward computers tend 
to be sensing and thinking which is in line with the 
Myers-Briggs Type Indicator. Students were also more 
Visual (92%) then Verbal (8%). This may be largely the 
result of age. Generation Y has grown up in a much 
more visually-oriented world and results may differ 
greatly if the population were not so homogeneous. 
Also not surprising is the all the students identified as 
being Sequential learners. This again may be a direct 
result of age. 

The Learning Styles Questionnaire (LSQ) developed by 
Honey and Mumford (1992) identifies four types of 
learners: Activists, Theorists, Pragmatists, and 

Reflectors. Students identified largely as Theorists 
(72%) and to a lesser degree as Reflectors (15%). This 
may be the result of their level of learning— this was a 
sophomore class and students at this level are not 
required nor have they developed skills in group 
dynamics and intuitive decision-making. Honey and 
Mumford indicate that individuals tend to rely on one of 
these approaches when they are engaged in learning. 
This seems to hold true here. 

Eachus’ (1993) Academic Self-Efficacy Scale assesses 
the extent to which students believe they have the ability 
to exert control over their academic environment. 
Students’ scores were all across the board here, some 
being very high and others relatively low. There does 
not seem to be a pattern with respect to gender or course 
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section. The small sample size may be a deciding factor 
and a larger sample size is needed to better understand 
the implications of self-efficacy. 

Thus, from this preliminary analysis we find that there 
are some scales that seem to better predict learning 
outcome than others. Solomon and Felder’s (1996) 
Index of Learning Styles (ILS) seems to have more 
consistent and applicable predictive value than the other 
scales. Kolb’s (1984) Learning Style Inventory also 
seems to shed some light on which learning style is more 
prevalent in distance learning. The results of this pilot 
study suggest that a larger sample might be useful in 
determining whether or not these scales do indeed 
predict learning outcomes. Correlation with students’ 
final grades might be even more of a significant factor. 
A follow-up study will be done to determine the effects 
of grade on learning style. 

IMPLICATIONS FOR EDUCATORS 

Given the sheer number of students who anticipate 
taking or are currently taking distance-learning courses, 
the need to be able to quickly and easily assess their 
potential level of success in these courses is paramount. 
Simply because it is an available alternative to the 
traditional classroom does not make it a viable option for 
everyone. Students have specialized needs and skills 
and not every student may be suited to a distance- 
learning environment. It is the responsibility of 
educators to make sure that students know and 
understand the risks and potential drawbacks to this 
environment. The last thing we want is for a student to 
be “lost in cyberspace” when a simple assessment early 
on might have identified the student as a poor candidate 
for distance learning. Regardless of the attractiveness 
and profitability of this new pedagogy, we must still be 
available for our students and provide them with every 
opportunity to further their education as they go forward 
on their journey in today’s fast paced digital world. 
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ABSTRACT 

Many educational institutions have recently started to move some of their course materials online, and then used the 
terms “flexible delivery " or “flexible learning” both in their everyday discussions and in their promotional literature . 
However, the use of asynchronous learning techno logies does not of itself make for true flexibility in learning. In 
almost all cases, the flexibility extends to location only. In very few cases does the flexibility extend to the method 
of learning or the order of the material to be studied; and in almost none does the flexibility extend to any temporal 
aspects. 

This paper suggests some ways forward for institutions seeking to make their courses truly flexible. 



INTRODUCTION 

A large number of universities and other educational 
institutions have recently been moving their course 
materials online, so that they may be used as the 
foundations for courses being undertaken by students 
remote from the campus, or by on-campus students as a 
supplement to face-to-face sessions, ora combination of 
both. More often than not these institutions will then use 
the terms “flexible delivery” or “flexible learning” both 
in their everyday discussions and in their promotional 
literature. However, the use of asynchronous learning 
techniques, such as placing notes on the Web, or 
arranging for the use of electronic discussion lists, does 
not make for true flexibility in learning. In almost all 
cases, the flexibility will be in location only; that is, the 
students are enabled to access the materials from 
anywhere with good internet connections. In very few 
cases does the flexibility extend to the method of 
learning or the order of the material to be studied ; and in 
almost none does the flexibility extend to any temporal 
aspects. Students must still enrol at a certain (often 
inconvenient) time, take assessment items at a certain 
(often inconvenient) time, and complete theirstudy at a 
certain (often inconvenient) time. 

And yet skillssuch as walking, orswimming, or driving, 
or playing chess, are all learnt without such fixed 



constraints. Indeed, in almost no other aspects ofhuman 
learning, except for courses at recognized academic 
institutions, are such stricttemporal limitations imposed. 

What does it mean for an institution to offer truly 
flexible learning? How does that differ from current 
practice? And what are the major obstacles thatneed to 
be overcome? 

CURRENT MODELS OF FLEXIBLE DELIVERY 

At its most minimal, the term flexible delivery can be 
taken to mean any form of delivery where students have 
a choice of study mode. This could mean, for example, 
that students are enabled to study from a distance via the 
provision of printed materials. 

More recently, the term has been used to describe almost 
any course that utilizes advanced learning technologies. 
For example, four examples of models of web-based 
delivery that are in current use for the delivery of 
computer science or information system courses have 
been described in [1], 

In the most minimalist of these four models, the naive 
model, face-to-face lecture notes are placed online; the 
course is then considered “flexible” in the sense that 
distance education students have access not only to pre- 
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distributed printed materials but also to online lecture 
notes. 

Even with the other three more sophisticated models, 
(the standard model, theevolutionary model [3], and the 
radical model [4], [5]), flexibility is enabled only in the 
sense that students unable to attend on-campus lectures 
and tutorials have a range of varied web-based facilities 
and email-based discussion groups to assist them to 
learn, and to enable greater interaction, both with the 
instructors and with other students. 

Occasionally institutions have experimented with the 
design and implementation of courses including a wider 
definitionof flexibility; in almost all cases, however, the 
experiments tend to have been with a single course, with 
a limited number of students, and limited scope [2]. 

So we must ask the question, to what extent are such 
models as those described above, and others like them, 
truly flexible? The current author has proposed a 
possible classification scheme for flexible learning, 
which would seem to indicate that most current courses 
do not rate very highly [6]. 

LIMITATIONS OF CURRENT MODELS 

While all of the models described above do indeed 
enable students physically situated at rem ote location s to 
study courses via the internet, true flexibility is denied 
because (a) studentsstill have to abide by a fixedmethod 
of study, (b) students still have to study the course 
materials in a particular order, and (c) students still have 
to abide by administrative timelines decided by the 
institution. 

Why Do Students Have to Abide by a Fixed 
Method of Study? 

The reasons here are primarily economic — to provide a 
variety of possible modes of study requires greater 
investment in time and materials. Giving students the 
choice of individual or group work, or learning from 
different texts, or (to take an extreme case) learning 
programming while allowing a choice of different 



programming language, has in the past been seen as just 
too hard and just too expensive. 

Why Do the Course Materials Have to Be Studied 
in a Particular Order? 

On occasion, this is because the nature of the material is 
such that concepts build one upon the other, and the 
particular sequence is largely determined by the nature 
of the material itself (for example, it may be thought 
desirable to learn about the properties of concrete before 
learning how to design concrete structures). However, 
more often the primary reason for a particular order is 
not the increasing complexity of the material, butrather 
the need to learn according to the order of the quizzes, 
tests, and other assessment items that are presented 
through the course (there is little point in learning about 
programming principles in week one and design 
principles in week two if the test on design occurs at the 
end of week one and the test on programming occurs at 
the end of week two). 

Why Do Institutions Have So Many Administrative 
Timelines? 

This is the primary obstruction to true flexibility, and 
here the primary reasons are many and complex — they 
would include reasons that are primarily historical, 
cultural, political, and economic. And, in large part, 
institutions in many countries are hamstrung by 
bureaucratic government regulations that require the 
reporting of detailed statistics, where the models used 
assume fixed timelines for enrolment, withdrawal, and 
completion. 

TOWARDS TRUE FLEXIBLE LEARNING 

In a learning environment which seeks to be truly 
flexible, students should be able to enroll when they 
want, study from any location at times of their own 
choosing, take assessment items when they feel they are 
ready, and complete as soon as they are able. 

For this to become a reality, all of the obstacles detailed 
in the earlier sections need to be overcome. Most 
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educational institutions are, by nature, fairly 
conservative and resistant to change. However, the 
rewards for those able to adapt themselves to the new 
environment are potentially large. So how should 
universities seeking to make their courses as flexible as 
possible set about making the necessary changes? 

The following steps are seen to be essential. 

• a rationalization in the modes of course delivery 

• a comprehensive redesign of course materials 

• a revision of the types of assessment used 

• a change to institutional timelines 

These four steps will be dealt with separately. None of 
the four are without difficulties. 

A Rationalization in the Modes of Course Delivery 

No institution seeking to compete in the modern global 
environment can afford to provide courses in three 
completely different formats, yet that is precisely what 
many have cho sen to do by default. Where once courses 
were provided only face-to-face, the need to attract 
distance education students has meant delivery also by 
means of printed materials; and the adventof the internet 
has meant delivery also online. 

As a result, the same lecture will often be delivered to 
one group of students in a live session by an instructor; 
again in printed form prepared prior to the start of the 
semester; and yet again in the form of an html page or a 
pdf document provided online via the Internet. 

This is, of course, not cost-effective. For the institution 
seeking maximum flexibility at a reasonable cost, the 
firsttwo of these methods should be dispensed with, and 
all efforts putinto effective and comprehensive delivery 
of the material online. 

Instructors will need to be available on a continuous 
basis. Thus, rather than having the responsibility forone 
or two courses per semester, in the new flexible 
environment it may be that an instructor has overall 
responsibility for one course for an extended period, 
perhaps even two to three years. The instructor can 



expect to have to field questions and comments from 
students continuously throughout this period. 

A Comprehensive Redesign of Course Materials 

Face-to-face sessions are necessarily sequential in 
nature. Printed chapters in a book are sequential too, but 
at least here the adventurous reader can skip ahead. 

No medium is better suited to a variety of possible orders 
than the Internet, with its built-in hyperlinks. Students 
are easily enabled to study the material in an order that 
suits them, rather than a single pre-ordained order laid 
down by the instructor. 

The materials should thus be designed on the basis that 
the order of study is uncertain. No longer should 
instructors assume that the student has studied Module 
4 prior to Module 5, northat the most complex material 
has to be placed at the end of the course. 

This means that the face-to-face lectures used for so 
many years are unlikely to form a good basis for 
effective delivery online. Instead, the materials will 
almost certainly needto be redesigned not only to make 
use of the specific strengths (hyperlinks, graphics, 
animations, etc) provided by the new medium, but also 
to take advantage of the fact that the Web enables 
simplified non-sequential access to the course materials. 

It is important to note, however, that in the truly flex ible 
environment, there is no longer a need for new course 
materials and assessment items to have to be delivered 
every semester. In fact, quite the opposite: materials 
should be deliberately designed to last for extended 
periods. Thus, many of the costs andstresses associated 
with traditional delivery methods may in practice be 
significantly reduced. 

A Revision of the Types of Assessment Used 

Difficult as both rationalization and redesign are, both 
pale into somedegree of insignificance when compared 
to the problem of assessment. In the conventional 
environment, each assignment item is normally due at 
some particular date and time, and all students attend 
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examinations simultaneously. If, on the other hand, 
students can take assessment items attimes oftheir own 
choosing, what implications does this have? 

First, there cannot be direct supervision. This would be 
both impractical and uneconomic. So some form of 
assessment must be devised which the student can take 
unsupervised. This means, in practice, that there cannot 
be any restriction on the materials to which the student 
is allowed access. 

Second, the assessment must be made available for 
completion at any time. No longer is it possible to plan 
based on two or three examination periods per year, for 
example. 

Third, students should not be put at an undue 
disadvantage because of any failure in technology. 
Thus, timed online tests, one of the most commonly-used 
fonns of assessment associated with advanced learning 
technologies, should be avoided wherever possible. 

What options does thatleave? The most obvious option, 
the setting of different assessment items for each student 
in a course, while desirable, is unlikely to be feasible. 

However, options do remain. In many conventional 
courses, take-home examinations are the norm. In 
others, students are given a list of possible examination 
questions at the beginning of the course, and informed 
that the examination will consist of a subset. Both of 
these point to possible ways forward. 

Since students can elect to take assessment items at any 
time, care must be taken to ensure that solutions to an 
assessment item made available on Tuesday cannot 
subsequently be used for an assessment item on 
Wednesday. 

The preferred method of assessment under such 
circumstances is likely to vary according a number of 
factors, including the nature of the course itself - for 
example, whether' the content is mainly theoretical or 
practical, and only guidelines can be given here. 



Options include: the use of open-book online quizzes, 
but only where these are untimed, or alternatives are 
readily available should the technology fail during the 
taking of a quiz, and questions are randomly selected 
from a large data-bank; tests which consist of a selection 
of essay-style questions (which students can choose to 
submit at any time, of course); and problem-solving 
tasks where students provide explanations as to their 
methods of solution. 

The use of online electronic submission enables the use 
of programs specifically designed to check for 
plagiarism and the copying of unreferenced materials, 
and it would seem likely that the use of such programs 
will assume much greater importance in truly flexible 
delivery. 

A Changes to Institutional Timelines 

Institutional timelines are generally of three types: 
timelines imposed by the instructor, normally for 
pedagogical reasons; timelines imposed by the 
bureaucracy of the institution, normally for reasons of 
administrative efficiency; and timelines imposed by 
outside agencies such as state or federal governments, 
normally for reasons of accountability and funding. 

In the truly flexible model, the first of the three, 
timelinesimposed by the instructor, should be dispensed 
with entirely. Students should be free to study at their 
own pace, according to their wishes and the demands of 
their other commitments, whether they be work-related, 
social, religious, or whatever. 

The second of the three, timelines imposed by the 
institution, should also be dispensed with. This implies 
that institutions need to reform their procedures so as to 
be able to accept enrolments into both programs and 
course at any time of the year, without the restriction of 
artificial deadlines. Similarly, successful completion of 
courses needs to be able to be recorded at any time. 

Both administrators and academics will be freed from 
the limitations of “terms” or “semesters” when students 
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are actively studying, and vacation periods when they 
are not. Undergraduate students will expect to be able to 
learn at any time, in much the same way that such a 
facility is taken for granted by PhD and postgraduate 
students. One consequence of this is that where 
possible, each course should becovered by a small team 
of academics, rather than by a single individual. 

The one deadline that institutions may be justified in 
retaining is the limit to the time a student can spend 
enrolled in a course without successful completion. 
Without such a limit, it would be possible forstudents to 
remain in a course effectively forever. This is not 
advantageous to the institution, for two reasons; firstly, 
the student’s presence in the course is likely to be 
expensive in terms of staff time and resources; and 
secondly, many courses change over time, and it would 
be inappropriate for a student to claim successful 
completion based on mastery of very old material (while 
one is happy to accept that a student passing 
Programming 101 in 1972 may well have learnt 
ALGOL, the same basis would not be acceptable for 
passing Programming 101 in 2002, for example). 

The third of the three timelines, those imposed by 
outside agencies, are the least easy for the institution to 
change, for obvious reasons. Nevertheless, it seems at 
least feasible that the institution seeking to make truly 
flexible learning a reality would be able to negotiate 
alternative reporting arrangements. The details of such 
arrangements would vary from country to country and 
even state to state, and are clearly beyond the scope of 
this paper to cover in detail. Nevertheless, the direction 
is clear: no longer should students be considered to be 
enrolled in particular courses for particular pre-defined 
semesters. That is, enrolment number should no longer 
carry with them implications about specific dates of 
enrolment, or expected dates of completion. 

SUMMARY 

This paper has indicated that so-called “flexible 
learning” is, more often than not, only traditional 
distanceeducation with printed materials replaced by, or 
supplemented with, material on the Web, and that the 
indicatedflexibility rarely extends beyond one particular 
aspect, the geographical location of the learner. 



True flexibility is achieved only when the use of 
advanced learning technologies is such as to enable 
learners to study attimes of their own choosing, without 
the necessity to abide by antiquated procedures and 
timelines. 

Some difficulties of truly flexible learning have been 
outlined, and some ways forward have been suggested. 
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ABSTRACT 

The rapid diffusion of information and communication technology (ICT) in many organizations, and in public 
universities in particular, in Kenya during 1980s and '90s, brought with it unprecedented challenges to these 
organizations. Not only is the technology new to use in many ways and, therefore, requires new skills on the part of 
users, but for the technical staff it demands from them completely new ways of managing ICT as opposed to those 
approaches required for managing other types of enterprise resources . The ICT resources continue to increase in 
numbers, value and sophistication as more and more organizations invest in the new technology, and with it the 
complexities of managing them become even more pronounced. This development also means that providing ICT 
services to an increasingly knowledgeable user base, whose requirements for, and expectations from, ICT centers 
rises continuously, is also becoming a very serious challenge. For the ICT management staff these requirements 
represent heavy responsibilities and daunting tasks that demand from them a sound knowledge of ICT management. 
However, lacking suitable models to apply leads them to adapt different approaches and styles to managing ICT, with 
serious consequences on the quality of ICT service, which leads to loss of confidence in the new technology by the 
user community. Recognizing this problem , a model for improving ICT management based on proven models is 
presented. The results from the case study indicate that the model can be a useful and practical tool to managing ICT 
effectively and efficiently, not only in public universities, but in other organizations in Kenya and other third world 
countries. 



INTRODUCTION 

Increasing amounts of hardware and software havebeen 
acquired in the last 10-15 years by an increasing number 
of organizations, and in particular, the public uni- 
versities, in Kenya. However, due to lack of research in 
the management of ICT prior to, during, and after the 
introduction of ICT in this part of the world, there are 
signs of growing problems associated with the influx of 
the new technology. Users have expressed frustration 
due to unfulfilled expectationsof the performance of the 
new technology. The technical staffs have adopt reactive 
and often chaotic and ad hoc rather than proactive 
approaches to managing ICT. As argued in (Wanyembi 
et al. 2000), expectations of users for improved services 
are not fulfilled due, in part, to low quality of 
management, control and maintenance of ICT. This 



leads to poor realization of the potential that can be 
derived from the investment in the new technology by 
the universities. The problem cited above can thus be 
stated briefly as follows. 

There is a lack of complete models for 
understanding and improving the manage- 
ment of ICT in public universities in Kenya, 
leadingto these institutions failing to realize the 
full potential of their ICT investments. In 
addition, there are inherent limitations in many 
ofthe existing models, as a result ofwhich these 
models fail to address situation-specific pro- 
blems found in many developing countries. 

The implicatio n of this statem ent is that while there are 
models that have been proven to work elsewhere in the 
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developed world, these models do nottake into account 
the situation-specific factors commonly found in third 
world countries, which differ, considerably from thoseof 
the developed world. The situational factors that 
characterizemuch ofthe third world countries where the 
infusion of ICT is growing at an unprecedented pace, 
includeage, size, location, level of computer literacy and 
communication infrastructure, A theoretical framework 
of these factors together with complexity factors 
associated with the new technology are presented in this 
paper, in which we adopt the modelling approach to 
describe ICT related issues in public universities in 
Kenya. We also present a 4-function model to improve 
an ICT situation in an organization from one level to 
another, A case study where this model was applied is 
also described and the practical implications of the 
results of the application are given, 

MODELLING ICT RELATED 
ISSUES IN AN ORGANIZATION 

A model can be defined as an abstraction of a concept, 
object, or event. One model thatcan be used to describe 
and manage information and communication technology 



is the management paradigm. The management para- 
digm, illustrated in Figure 1 (Looijen 1998), provides a 
top-level view of concepts of an organization with regard 
to ICT related issues. 

The term paradigm is used to characterize a worldview, 
a lens through which we are able to identify and focus on 
key issues (Glazier 1992). Essentially, the management 
paradigm consists of three basic entities: th eReal System 
(RS), the Information and Communication Technology 
(ICT), and the Management, Control, and Maintenance 
of ICT ( MCM ), Additionally, the Relationshipsbctween 
the entities and the external Influences that act on the 
organization constitute vital parts of the paradigm for 
inclusion due to the role they play. 

In modelling issues in a university, we can adopt the 
systemsperspective on organizations and organizational 
processes. Acco rding to this pers pecti ve , an organizatio n 
can be viewed as a system of related objects (Ackoff 7 1 ; 
Checkland 1981; and Sol 1982). In the same organiza- 
tion can also be found people (actors), equipment and 
materials used (objects) and activities that they perform 
(processes). 



FIGURE 1 

THE MANAGEMENT PARADIGM 
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In the entity Real System (RS) critical issues related to 
ICT are the user requirements for ICT resources and 
services, that users demand from their organizations in 
order to support their day-to-day operational activities 
and processes. From the point of view of users in their 
working environments, the most important requirem ents 
are the availability of ICT resources and services, the 
functionality and flexibility of ICT, the maintainability, 
the performance, the reliability, and the security, of ICT. 
These requirements are prerequisite to effective 
performance of users and ultimately their organizations. 
User requirements exert pressure onthe organizations to 
adopt an integral as opposed to an isolated approach to 
ICT provision in order to satisfy the demands from 
students and staff and other interested parties, who are 
collectively referred to in this paper as users. Thus, in 
orderto realize optimal benefits, the organization should 
strive towards synergy, transparency and sharing ofICT 
resources to ensure greatest exploitation, while users 
strive towards autonomy to control the availability, 
reliability, and security oftheir own ICT resources (Wijs 
1995) in their respective workplaces. 

In the same entity RS, preconditions such as information 
policy and planning, policies on centralization or de- 
centralization of activities, finances and personnel, 
dictate to a large extent the manner in which ICT usage 
can be realized. Additional preconditions include 
policies on concentration or de-concentration ofICT 
resources, safety of users and equip ment, stan dardizati on 
of hardware/software, and service level agreements 
(SLA). 

Existing situational factors under which technical staff and 
users operate are critical aspects to consider during the 
implementation ofICT projects in organizations because 
situations differ from organization to organization and 
have been shown to cause risks (Euromethod 1994). The 
situational (or contingency factors) include the 
organization’s age, size, (Mintzberg 1979), location, 
technology environment, organizational culture, and 
general communication infrastructure. These factors must 
be recognized for the impact they have on organizations 
and users and accordingly must be taken into account 
during implementation, utilization and management of 
ICT. The issues and factors cited above can be effectively 
modeled in the RS entity of the paradigm. 

In the entity ICT, the technology can be modeled to 
include the components, the states in which it exists, and 
the complexity factors within which it is utilized, 
exploited and maintained. The performance ofICT is 
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limited by the characteristics ofICT imposed upon the 
users. These characteristics include the speed of 
processors, memory size, and capacities ofcommunica- 
tion lines. Our perception of ICT is based on the 
definition ofICT given by (Brussard 1980; Sol 1982, 
and Looijen 1998), that is, as all the hardware with the 
relevant basic software and application software, 
datasets, procedures, and persons, involved in the control 
or support of the real system or business processes. As 
Land and Hirschheim emphasize, itis fruitful to think of 
ICT as not a goal in itself, but as social systems: a 
complex web of interpersonal relations that produces, 
utilizesand communicates information (Land 1 983). As 
distinct and separate components, ICT exhibits indi- 
vidual characteristi cs, which remain practically constant 
over a period of time. These characteristics are termed 
static to distinguish them from dynamic characteristics 
that are associated with implemented ICT. The concept 
of performance and behavior characteristics ofICT was 
first introduced by van Hulzen and De Moel (Hulzen et 
al., 1987), from which a framework of concepts such as 
availability, continuity, integrity, accessibility, and 
timeliness was formulated. Then Delen and Rijsenbrij 
(Delen et al. 1990) made a distinction between static 
product characteristics and dynamic product 
characteristicsofICT. Static product characteristics such 
as processor speed, memory size, are considered relevant 
to technical and operational staff while dynamic product 
characteristics such as availability, responsiveness of 
ICT, are concerns of the user environment. 

Extended State Model 

Modeled in the entity ICT is the Extended State Model 
(ESM), which essentially describes the life history cycle 
ofICT and the states in which ICT may exist at any given 
point in time. According to Looijen, The Extended State 
Model , see Figure 2, consists of the states information 
policy and planning (IPP), developm ent (D), acceptance 
and implementation (Al), utilization (Ul, U2), 
exploitation (El, E2) and maintenance (Ml, M2). 

FIGURE 2 

THE EXTENDED STATE MODEL 
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In this paper thestatesU, E, and M (shaded) are relevant 
and will be focussed upon as they relate to user 
requirements more directly than other states. The state 
utilization (U) involves the users of ICT in their day-to- 
day activities as they make use of the functions of the 
information systems. At the organizational level, the 
state exploitation (E) takes place when the organization 
takes advantage of ICT to realize its goals. Whenever 
there are minor problems or modifications to be 
addressed concerning users and ICT, these are brought 
to maintenance (Ml) while major modifications 
regardingthe extension ofICT for greater exploitation of 
ICT from the phase E2 are addressed in the state 
maintenance (M2). 

Complexity Factors 

Many users and management staff oftenregand ICT as a 
complex system. Understanding how itfunctions under 
the prevailing circumstances presents itself as a special 
problem. Most importantly, comprehending how to 
manage, control and maintain ICT is not always 
straightforward. Often long periods of training and 
experience are needed to master some parts of ICT 
management. This special group of problems is referred 
to as complexity factors. A total of eight complexity 
factors associated with ICT management can be 
identified. These factors relate to the separate ICT 
components, their interrelations and their usage. The 
eight complexity factors are number ofICT resources 
(quantity) which the technical staff are in charge of. The 
resources can differ in type, make or origin (diversity), 
and be centralized or de-centralized to a high degree 
(distribution), and regularly be subjected to changes 
(dynamics). Other complexity factors are the variety of 
functions that ICT can have and which need to be 
mastered by users and technical staff alike (func- 
tionality), and which are supplied by a number of linked 
components (coherence). Finally, in a large university, 
different owners (ownership) may possess the ICT 
components and users can have varying requirements 
and stipulate different preconditions on their usage 
(usage or utilization). 

Forms ofICT Management and ITIL Processes 

The entity MCM can be described as entailing the 
management, control and maintenance of implemented 
ICT in accordance with the requirements and precon- 
ditions imposed by the utilization, the situational factors 
and characteristics ofICT com ponents. The entity offers 
ICT management services and positively influences the 



goals of the organization. Three forms of MCM can be 
distinguished: the functional management (FM), appli- 
cation management (AP), and technical manage-ment 
(AM). Their common structure can also be modeled 
using Mintzberg’s logo (Minzberg, 1979). These forms 
are depicted in Figure 3. We define the three forms of 
management as follows. 

Functional management. This involves managing the 
functional specifications and the functionality ofICT, 
i.e., all management tasks that are necessary to support 
and advise user environments within the framework of 
the utilization of ICT. The level of ICT utilization 
determines to a large extent the functionality ofICT. 

Application management. This involves the manage- 
ment, control and maintenance of application software 
and database structures (Wijs 1995, Looijen 1998). 
Changes can be initiated by faults reported by users, the 
need to extend the functionality of the software. The 
latter is referred to as application software maintenance. 

Technical management. This involves managing the 
hardware, the technical infrastructure and the systems 
software, i.e., all the tasks that are necessary to 
implement, to accept, and to operate the technical infra- 
structure. Optimization of the technical infrastructure as 
a consequence of faults , expansion or replacement is also 
part of technical management. 

Within each of the three forms in the entity MCM are 
twelve task areas distributedin fivebasic elements of the 
organizational structure — the strategic apex, the middle 
line, the operating core, the techno-structure and the 
support staff. The task areas are listed in Figure 3 and 
bear a relationship with Information Technology 
Infrastructure Library ( ITIL) processes. 

The Information Technology Infrastructure Library 
(ITIL) is identified as a framework of the n, the best 
practices and a standard of IT service quality that 
customers should demand and providers should seek to 
supply' (CCTA 1999).” The ITIL provides a total of 9 
sets ofICT management processes, popularly known as 
the best practices’. These sets are: manager 's set , service 
support, service delivery, software support, networks, 
computer operations , environmental strategy, 

environmental management, and office environment . 
Two of these sets(service supportand service delivery) 
were selected for application in this study in order to 
focus on the most suitable ICT processes that relate to 
user requirements for quality ICT services. The two sets 
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FIGURE 3 

FORMS OF MANAGEMENT AND ITIL PROCESSES 



Key: Task Areas 

SM Strategic Management 
TM Tactical Management 
OM Operational Management 
PM Personnel Management 
GBS General Business Support 
TSu Teehn ical Supp ort 



UM Utlization Management 
FM Functional Maintenance 
AM Application Maintenance 
OC OpcrationalControl 
MTI- Maintenance Technical 
OS Infrastiuctuc and Operational 
TSe Technical Services 



are service support and se rvice delivery. Service Suppo rt 
processes include Change Management(CHG), 
Configuration Manage ment (CON F) , Help Desk (HLP), 
Problem Management(PRB), and Software Control & 
Distribution(SCD) while Service Deliv ery processes are 
Availability Management (AVLM), Capacity Manage- 
ment (CAPM), Contingency Planning (CONT), Cost 
Management(CSTM), and Service Level Manage- 
ment(SLM). A typical ICT center that implements the 
two sets of ITIL processes would look like illustrated in 
Figure 4, in which, for example, 

(a) a user makes a call to the help desk (first line sup- 
port) to report an incident or malfunction related to 
hardware/software. 

(b) if the incident cannot be resolved at this point, the 
incident is referred to the process problem 
management. The problem management process 
(second line sup-port) investigates the incident to 
determine the underlying cause. In this process, a 
request for change (RFC) may be made if necessary, 
calls in capacity management to assist. Henceforth, 
the incident is referred to as a problem . Service level 



management is informed that a SLA has been 
violated. 

(c) change management process co ordinatesthe request 
for change (RFC), which runs in parallel with the 
process configuration management(see below). 
Change management process determines the impacts 
of the change on both the quality of existing ICT 
service as well as on the service level agreements. 
This relates change management to Service level 
management process. 

(d) cost management process assists, through service 
level management process, with the business cost 
justification for hardware/software upgrade with 
suppliers, who then provides new ICT compo nents 
or services to provide an answer/solution to the 
problem. 

(e) contingency planning process becomes involved in 
change management process either to ensure 
recovery of services or maintenance of ICT service 
at acceptable levels as per SLAs. 
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FIGURE 4 

MODELLING PROCESSES AT AN ICT CENTER 



(f) software control and distribution (SC&D) controls 
the implementation of the change by rolling out he 
replacement hardware and software. SC&D also 
updates configuration management with details of 
new releases and versions. 

(g) availability management process becomes involved 
in considering hardware/software upgrade to ensure 
it meets the required availability and reliability 
levels. 

(h) capacity management, availability management, 
contingency planning, and cost management pro- 
cesses provide vital data on cost, quality of ICT 
services, and impacts due to changes, for 
configuration management process. These four 
processes are also involved in enactment of service 
level agreements between ICT organization and 
users/suppliers. 

(i) configuration management process ensures CMDB 
information is upgraded throughout. 

Finally, in the entity MCM, the capability maturity 

model is used to give the levels for improving ICT 



management processes. The Capability Maturity Model 
(CMM) developed by Software Engineering Institute 
(SEI) ofCarnegie Mellon University describes the stages 
or plateaus through which organizations evolve as they 
define, implement, measure, control, and improve their 
processes (Paulk etal. 1993). However, for this model to 
be useful in this case, the CMM levels must be re- 
defined in terms of ICT management processes rather 
than in terms of software processes. The first three of 
these levels used in this paper are defined as follows: 

Level 1 (Initial). At this level there are no management 
processes and reaction to events happens in an ad hoc 
manner. The management is completely in the hands of 
operational “authorities” who decide what has to be 
done. The workload is most of the time high, and the 
work is carried out in an uncoordinated way. The latter 
doesn’t necessarily mean that all efficiency is ignored. 
The point is that it cannot be measured, because each 
separate realization is strongly influenced by the 
individuals who carry it out. 

Level 2 (Repeatable). At this level the work is done in 
a processes-like manner, but there are no formal process 
des criptions. The management recognizes the impor- 
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tance of processes like incident management, problem 
management, change management, configuration man- 
agement and software control and distribution, and 
carries them out in a pragmatic way. The decision- 
making with regards to this is mostly inspired by the fact 
that similar processes in other organizations or in other 
situations seemed useful and looked susceptible to 
repetition, hence the name repeatable. 



in the entity MCM (i.e., ICT supports 
MCM). 

MCM— »ICT personnel staff in the entity MCM 

manage the information and com muni- 
cation technology resources in the 
entity ICT (i.e ., MCM uses or manages 
ICT). 



Level 3 (Defined). At this level the management 
processes are documented and standardized. They are 
related to the Service Level Agreements (SLA), which 
are established between themanagement and the users of 
the services. This implies that the services of the 
management are more predictable and that action is 
taken in a structured way on issues like performance 
improvem ent, proble m res olution , transport of necessary 
data and data storage capacity. Due to this, the service 
users consider the ICT management as a service 
organization. 

Modelling Relationships Between Entities 

The importance of modelling relationshipsbetween and 
within entities (See Figure 1) in the management 
paradigm lies in its ability to reveal the strengths and 
weaknesses of the various parts of an organization with 
regard to ICT related issues. These relationships are key 
to identifying and understandingthe problems associated 
with the management of ICT in an organization. 

In summary, the following relationships between the 
basic entities are defined (the arrow — » indicating the 
relationship defined between entities). 



RS— >ICT users in the entity real system impose 

requirements for resources in the entity 
ICT to enable them (users) to perform 
their day-to-day activities, (i.e., RS 
exploits ICT). 

ICT — > RS information and communication tech- 

nology resources in the entity ICT 
support the activities in the entity RS 
through enhancing effectiveness and 
efficiency (i.e., ICT supports RS). 



ICT— »MCM information and communication tech- 

nology resources in the entity ICT 
provide useful information required for 
ICT management to the personnel staff 



RS— »MCM the entity RS employs the entity MCM 

to manage its information and com- 
munication technology resources. 

MCM— »RS technical staff in the entity MCM 

responds to requests in RS). 

The above six relationships play a significant role in 
effective and efficient management of information and 
communication technology. Problems related to ICT in 
organizations can easily be traced to breakdowns in 
relationships between the relevant entities. 

Modelling External Influences 

Acting on an organization, such as a university, are the 
external influences (hereafter simply referred to as 
influences). Strictly, influences ought to be considered as 
being part of the real system, but due to their external 
nature, they are treated in this study as a separate entity. 
Because the organization receives materials, resources, 
students, employees, system of laws, finances, and so on 
from the external environment, and gives back to the 
same environment trained manpower, feedback infonna- 
tion in form of reports, financial returns, research 
findings, and so on, this entity must necessarily be 
treatedseparately from the other entities. Any significant 
perturbations in the external world is bound to impact on 
the organization, either positively or negatively, 
depending on the source, nature and magnitude of the 
influence. Among the influences identified for inclusion 
in this study are managerial, donor, technological, eco- 
nomic and cultural influences. 

Having thus modeled ICT related issues, by incorporating 
various models, in the university we present a model to 
improve the management of ICT in the next section, 
bearing in mind the role the other issues play in the overall 
scheme. In this paper we will limit our improvement to the 
user requirements in the entity RS, and the MCM 
processes in the entity MCM, against the background of 
other ICT related issues described in the paradigm. 



32 



Proceedings of the 16 h Annual Conference of the International Academy for Information Management 



FUNCTIONS OF ICT IMPROVEMENT MODEL 

The ICT management improvement m odel, is presented 

in Figure 6, and consists of four functions. 

(a) Depiction of the 1ST (German word for current 
situation)-in the entity RS (user requirements), and 
in the entity MCM (forms of ICT management , 
service support and service delivery processes f 
Based on the criterion existence , these issues are 
indicated according to whether they exist (in which 
case a value 1 is given) or not (in which case a value 
0 is given), 

(b) Qualification of the current situation-issues in the 
respectiveentities depictedin (a)are assigned values 
based on the values 0 = not availab le, 1 = LOW, 2 = 
MEDIUM, 3 = HIGH criterion for effectiveness. 
This structured approach is based on the observation 
that IS (effectiveness) issues are qualitative in nature 
(Kanungo et al. 1999). In the case of ICT 
management processes, attributes, , at Levels 2 
and 3 are given weighting, W { , that varies from 1 
(for least important) to 3 (for most importantissue). 

Attributes at Level 2 (and weights, Wj): 

If the process is: 



on schedule (1) 

cost-effective (1) 

planned (2) 

is practiced (2) 

trained (2) 

enforced (2) 

measured (2) 

process-like (3) 

recognized (3) 

performed pragm atically (3) 

repeat earlier processes (3) 



Attributes at Level 3 (and 


weights, Wj) 


If the process: 




has preparedness criteria 


(i) 


is documented 


(i) 


is standardized 


(2) 


is peer reviewed 


(2) 


has completion criteria 


(2) 



is related to service criteria (3) 

has predictable services (3) 

has structured actions (3) 

is seen as an organization (3) 



The capability maturity level for a process, L, is 
obtained by summing up all the values of the 
attributes calculated from the equation: 

L = 2(W, . XJ/IW (1) 

It is clear from the equation that a process can take 
on rational values ranging from 0.0 to 3.0. However, 
by convention, all processes thathave values below 
1 ,0 are automatically assigned a value 1 .0. It is also 
clear that a process can have attributes at all levels, 
but fail to achieve the highest level if, for example, 
not all the attributes are not rated HIGH (3). Hence 
each process is examined across the entire spectrum 
of attributes, given appropriate values and from 
these the capability level for that process is 
calculated. The weights assigned to attributes may 
be varied according to the person performing the 
assessm ent. 

(c) Definition of the SOLL (German word for future 

situation)-future situation for each of the issues 
depicted and qualified in (a) and (b), respectively, is 
described in terms of what the improved situation 
should be like. The list of attributes is checked for 
those attributes that do not score the highest value. 
For example, if the attribute practice at Level 2 
scores a value, 1, it implies that there is a need for a 
higher standard of practice in order to achieve the 
current level (2). This becomes the desired 

situtation and in effect defines the SOLL situtation 
for the attribute fo the preocess in question 

(d) Transformation is the realization of the new. 
situation from the current (1ST) situation. 

This function attempts to realize the new situation 
defined in (c) for each process. Clearly, an 
appropriate approach is required to transform any 
situation into a new and better situation than the 
existing one. Plan of activities must be devised, 
assignment of responsibilities made, resources such 
as funds, equipment, staff, and time needed must be 
acquired, and actual activities performed in order to 
bring about the new situation. The process takes into 
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FIGURE 6 

ICT MANAGEMENT IMPROVEMENT MODEL 



account the prevailing situational and complexity 
factors, and the external influences, discussed in section 
on modeling, with the objective of formulating the 
strategies needed to minimize their impact. Particular 
attention is paid to the relationships between entities to 
detennine what role, direct or indirect, actors in each 
entity can play. 

THE CASE STUDY: 

UNIVERSITY OF NAIROBI, KENYA 

The case study chosen forthe application of this model, 
University ofNairobi (www.uonbi.ac.ke), is the oldest 
and largest of the six public universities in Kenya. 
Situated in the capital city, the university has a total 
student population of about 15,000 and comprises seven 
campuses scattered in and around the city. The campuses 
were the Main, Choromo, Lower Kabete, Upper Kabete, 
Kikuyu, KenyattaNational Hospital, and Parklands.This 
geographical dispersion presents the university with a 
daunting task in providing its large number of students 
and staff with the ICT resources and services that they 
require, especially networking. However, its location 
also exposes it to the rapid global technological changes 
compared to other institutions and gives it the 
advantages of communication and other infrastructures 
not enjoyed by the other universities. 

Three embedded sub-case studies were selected, 
according to (Yin, 1994), for the application of this 
model. 



• the University Library IS (ULIS), which has network 
of five branches 

• the University Network Backbone (UNB) 

The Joint Admissions Board has an information system, 
located in Chinomo campus, that processes the admis- 
sions of undergraduates into academic programs in all 
the six public universities in Kenya. The JKML is the 
University Library that comprises five branches with the 
main library beinglocated in themain campus. It has an 
information system that students use to access Internet 
and perform searches of remote databases. The Uni- 
versity Network backbone located in the Chiromo 
campus is responsible for developing andmanaging the 
university network to enable students and staff to 
communicate internally and externally. The Institute of 
Computer Science coordinates all the technical work for 
the three units. 

The method used included presenting the model to the 
heads of the units concerned and discussing with them in 
detail each of the issues contained in the model. A 
schedule for meetings was arranged covering several 
days during the period August-September 2000. The 
four-functional model (depiction, qualification, defini- 
tion and transformation) was covered in detail and each 
issue qualified jointly by both the researcher and the 
officials concerned. The results obtained, therefore, 
represent qualitative descriptions of the situations 
covered. The results were checked, tabulated, and ana- 
lyzed, see Tables 1-5. 
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DEPICTION OF THE 1ST SITUATION 



Key: Existence of issue indicated by V Non-existence of issue indicated by 0 

TABLE 1 

DEPICTION OF USER REQUIREMENTS 



User requirements 


Existence of formulation of requirements in sub-case studies 




JABIS 


ULIS 


UNB 


Availability of ICT 


V 


V 


V 


Flexibility of ICT 


V 


V 


V 


Maintainability of ICT 


V 


V 


V 


Perfomtance of ICT 


V 


V 


V 


Reliability of ICT 


V 


V 


V 


Safety of people and equipment 


V 


V 


V 



TABLE 2 

DEPICTION OF MCM PROCESSES 



MCM processes 


Existence of MCM processes in sub-case studies 




JABIS 


ULIS 


UNB 


Functional management (FM) 


V 


V 


V 


Application management (AM) 


V 


V 


V 


Technical management (TM) 


V 


V 


V 


Change management (CHG) 


0 


0 


V 


Configuration management (CONF) 


V 


V 


V 


Help desk (HLP) 


V 


V 


V 


Problem management (PRB) 


V 


V 


V 


Software control & distribution (SCD) 


V 


V 


V 


A vail ability management (A VLM) 


V 


V 


V 


Capacity managemen t ( CA PM) 


V 


V 


0 


Contingency planning (CONT) 


0 


0 


V 


Cost management (CSTM) 


V 


V 


0 


Service level management (SLM) 


0 


0 


0 



QUALIFICATION OF THE 1ST SITUATION 
Low requirem ent 1 Medium requirement 2 High requirement 3 

TABLE 3 

QUALIFICATION OF USER REQUIREMENTS 



User requirements as an indication of 


Level of expression of requirements in sub-case studies 




JABIS 


ULIS 


UNB 


Availability of ICT 


3 


3 


3 


FI ex i bi l ity of ICT 


3 


3 


3 


Maintainability of ICT 


3 


3 


i 3 


Performance of ICT 


3 


3 


3 


Reliability of ICT 


3 


3 


3 


Safety of people and equipment 


3 


3 


3 
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Low qu ality process 1.0 -1.9 



Medium quality process 2.0 - 2.9 High qua lity process 



3.0 



TABLE 4 

QUALIFICATION OF MCM PROCESSES 



MCM processes 


Qualification of MCM processes in sub-case studies 




JABIS 


ULIS 


UNB 


Functional management (FM) 


1.0 


1.1 


1.1 


Application management (AM) 


1.4 


1.0 


1.0 


Technical management (TM) 


1.0 


1.0 


1.0 


Change management (CHG) 


1.0 


1.0 


1.0 


Configuration management (CONF) 


1.0 


1.1 


1.1 


Help desk (HLP) 


1.2 


1.0 


1.0 


Problem management (PRB) 


1.0 


1.0 


1.0 


Software control & distribution (SCO) 


1.0 


1.5 


1.5 


A va liability management (A VLM) 


1.0 


1.0 


1.1 


Capacity management (CAPM) 


1.0 


1.0 


1.0 


Contingency planning (CONT) 


1.0 


1.0 


1.0 


Cost management (CSTM) 


1.0 


1.3 


1.3 


Service level management (SLM) 


1.0 


1.0 


1.0 



The values for each sub-case study were obtained and 
compared. They showed that in almost all sub-cases, the 
standard of ICT management processes were very low 
(Level 1). 

For the sub-case study JABIS, 4 processes (TM, CHG, 
CONT, SLM) out of 13 representing (31%) of the 
processes were not performed at all . Of the remaining 9 
processes, 81.8% of their attributes were rated at LOW 
(1), while 18.2% of the attributes were rated at 
•MEDIUM (2). No attribute was rated HIGH (3), 

In the sub-case ULIS, a similar picture emerged with 2 
processes, (CONT, SLM), representing 1 5.4%, notbeing 
performed at all. Of the remaining 1 1 processes (84.6%), 
74.8% had their attributes rated at LOW (1), 22.1% at 
MEDIUM (2), and only 3.1 % rated at HIGH (3). 

In the sub-case study UNB, 2 processes (CONT, SLM) 
out of 13 processes representing 15.4%, were not 
performed at all. Of the remaining 1 1 processes (84.6%), 
had 74.4% of their attributes rated at LOW (1), 23.8% 
rated at MEDIUM (2), and only 1.8% rated at HIGH (3). 
The overall capability levels are given in Table. By 
convention all non-existent processes and those that fall 
below level 1.0 are regarded as being at Level 1. 



DEFINITION OF THE SOLL SITUATION 
User Requirements 

Since the level of requirements was high among users, 
and it was desirable that it remained high, the future 
situation was defined as the same level as the current 
one. That is the user awareness should always remain 
high. 

The SOLL situation for ICT management processes, 
indicated in brackets, are shown in Table 5. 

In the sub-cases JABIS, ULIS, and UNB, the processes 
that were not performed (TM, CHG, CONT, SLM), 
needed to be recognized, adopted and implemented as a 
priority due to the importance of the processes. This 
level of these processes should be at Level 2 at least. 
With those processes that were beingperformedat LOW 
(1) level, the future situation should be enhanced 
performance through careful planning, practice, training, 
enforcement, measuring, and performing activities in a 
process-likemanner. Additional ways include giving the 
processes more due recognition by authorities as well as 
learning from earlier successes. At a higher level, there 
was need, before performing any process, to have a 
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TABLE 5 

DEFINITION OF SOLL SITUATION FOR MCM PROCESSES 



MCM processes 


Definition of SOLL situation for MCM processes in sub-case studies 




JABIS 


ULIS 


UNB 


Functional management (FM) 


1.0 (2.0) 


1.1 (2.0) 


1.1 (2.0) 


Application management (AM) 


1.4 (2.0) 


1.0 (2.0) 


1.0 (2.0) 


Technical management (TM) 


1.0 (2.0) 


1.0 (2.0) 


1.0 (2.0) 


Change management (CHG) 


1.0 (2.0) 


1.0 (2.0) 


1.0 (2.0) 


Configuration management (CONF) 


1.0 (2.0) 


1.1 (2.0) 


1.1 (2.0) 


Help desk (HIP) 


1.2 (2.0) 


1.0 (2.0) 


1.0 (2.0) 


Problem management (PRB) 


1.0 (2.0) 


1.0 (2.0) 


1.0 (2.0) 


Software control & distribution (SCD) 


1.0 (2.0) 


1.5 (2.0) 


1.5 (2.0) 


Availability management (A VLM) 


1.0 (2.0) 


1. 0(2.0) 


1.1 (2.0) 


Capacity management (CAPM) 


1.0 (2.0) 


1.0 (2.0) 


1.0 (2.0) 


Contingency planning (CONT) 


1.0 (2.0) 


1.0 (2.0) 


1.0 (2.0) 


Cost managetrtent (CSTM) 


1.0 (2.0) 


1.3 (2.0) 


1.3 (2.0) 


Service level management (SLM) 


1.0 (2.0) 


1.0 (2.0) 


1.0 (2.0) 



preparedness criteria, document the processes, 
standardizing the processes, and peer reviewing the 
processes. Additionally, there should always be a 
completion criteriawhenever aprocess is completed. All 
these activities should be aimed at capability level 2 in 
order to bridge the gap that exists between the level of 
user requirements (3) and the current level at which the 
processes are being performed. 

TRANSFORMATION FROM 1ST 
TO SOLL SITUATION 

From the table ofqualification it become clear that most 
processes were rated at Level 1 m ainly because most of 
these processes were not established, or were being 
performed at LOW level. A process to transform the 
currentsituation to the future situationas defined above, 
was proposed. It consisted of three important steps. 

• selecting the issue to be transformed 

• considering the future situation in relation to the 
current situation 

• defining the criterion for achieving the goal 

• determining the resources needed to transform the 
situation 

• planning the transformation activities 

• acquisition of the resources needed 

• carrying out the activities required 

• comparing the results with the criterion set 



The results of transformation were in most cases incon- 
clusive for the simple reason that the time needed to 
carry out the transformation activities was short. Of 
greater importance, however, were theresources needed 
to transform a situation, including funds for training and 
equipment. Only in a few cases was it possible to 
achieve any tangible results. The authorities, however, 
recognized the value ofthe model and adopted it foruse 
in their respective sub-cases. 

SUMMARY AND CONCLUSION 

The model presented in this case study can be sum- 
marized as follows. 

(a) The model created awareness of vital ICT 
management and related issues and served a medium 
of communication among the stakeholders, thus 
facilitating cooperation among them, which had 
been lacking. 

(b) The model enabled the officials concerned and the 
researcher to identify the ICT management processes 
that were either non-existent, or, if they were, 
describe and qualify them as per the model. The 
future situations were and even define key issues 
related to ICT management in the university that 
they required. 
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(c) Using the adapted CM M framework, itbecome clear 
that, despite being aware of ICT management 
processes, the management of ICT in public 
universities is still atthe Initial level (Level 1). Also 
there was a wide discrepancy/gap between what the 
users require (HIGH 3) and what the ICT services 
are provided(LOW 1) by a factor ofthree (3) on the 
average. The challenge is to bridge the gap by 
improving the ICT management. 

(d) The units generally lacked the resources (funds, 
trainingopportunities, materials, equipment) neces- 
sary to improve their ICT management processes 
from their current situations. 

The model presented in this paper is a combination of 
established models adapted fo r applicatio n in the Kenyan 
public university context. Its primary purpose IS to 
create awareness and establish a basis for communica- 
tion among stakeholders (management, technical staff 
and users) on matters concerning ICT management. Its 
application is an attempt to come to terms with a 
growing problem of management of ICT in an 
increasingly complex situation brought about by the 
rapid influx of information technology into a world that 
was largely unprepared. It can be concluded that the 
model presents a practical solution to the emerging 
challenges of the new technology in Kenya and much of 
the third world countries. 
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ROBOCUP: 

MULTI-DISCIPLINARY 
SENIOR DESIGN PROJECT 



Kevin Lee Elder 
Ohio University 



ABSTRACT 

A cross-collegeteam of educators has developed a collaborative, multi-disciplinary senior design course at the University . 
This course offers an attractive opportunity for students from a variety of disciplines to work together in a learning 
community to accomplish a challenging task. It provides a novel multi-disciplinary learning environment where they 
will be encouraged to thinkabout issues related not only to their own major, but mo re global issues that are extremely 
relevant to real-world design situations. The project has the potential to enhance and extend the reputation of the 
University as a leader in equipping Mechanical and Electrical Engineers, Computer Scientists, an d Business students 
for the fast-paced, dynamic, team-oriented careers that are becoming the norm in ind ustry. The proposed project for 
this activity is designing, constructing, programming, and controlling a team of robots for the international RoboCup 
competition. This event, featuring mobilerobots competing in the game of soccer, promises to provide the U niversity 



high-visibility, positive exposure to help recruit quality 
Technology is currently seeking industrial sponsorship fo 
in a proposal to the NSF Education Directorate. 

OBJECTIVE 

When students graduate from the University they are 
equipped with skills relevant to a particular discipline. 
However, their education rarely prepares them for the 
situation that many encounter in the workplace: 
participation in multi-disciplinary teams. To enhance the 
undergraduate educational experience, the faculty 
involved propose to develop a multi-disciplinary senior 
design course within the College of Engineering & 
Technology and the College of Business. 

The proposed multi-disciplinary senior design course 
involves curricular innovation, building a new multi- 
disciplinary learning community. Although an initial 
goal of the course is to enhance and extend senior design 
courses, the project focus will enable faculty to also 
transferthe collective research results into undergraduate 
education and to involve undergraduate students in 
research activities. The faculty involved intend to 



indergraduate students. The College of Engineering & 
the proposed RoboCup team. Project results will be used 



leverage the existing funding and demonstrated 
institutional commitment by submitting an NSF 
Education Proposal. Finally, the proposed project, 
RoboCup (mobile robots competing in the game of 
soccer), will bring positive recognition to the University 
via the team of students competing in this highly 
publicized, international competition. This will enhance 
the ability to recruit quality students to the University. 

BACKGROUND 

Professors from the School of Electrical Engineeringand 
Computer Science, the Department of Mechanical 
Engineering and the Department of Management 
Information Systems have held meetingsto considerthe 
problem of educating students for success in multi- 
disciplinary team environments. The results of the 
discussions are (1) an analysis of the problem, (2) the 
means by which the faculty involved should test whether 
a proposed solution solves the problem and (3) a 
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proposed solution. The remainder of this paper presents 
the results ofthe discussions and the plan to implement 
the solution. 

The problem of educatin g students fo r success in multi- 
disciplinary team environments is one that needs to be 
addressed. Professional engineers are often confronted 
with the challenges of constructing, modifying and 
maintaining very large and complex systems, such as 
aircraft, spacecraft, ships and automated factories. 
Without exposure to the broad range of skillsrequired by 
such challenges, engineers will not know how to 
approach them in the most effective way. For example, 
an electrical engineer tasked with developing an 
autonomous mobile robot foran automated factory may 
attempt to design and construct the entire robot herself. 
Such an approach is not likely to be as successful as one 
in which a mechanical engineer is involved with the 
motor, drive train and platform aspects and a software 
engineer contributes the algorithmic designs. 

The current curriculum within the College of 
Engineering & Technology does not prepare students as 
well as it could for multi-disciplinary environments. 
Rather, depth and breadth are provided within a single 
discipline. For example, a software engineer learns: 

• How to program a computer in several programming 
languages; 

• Concepts for developing a broad range of specific 
software systems such as databases, operating systems, 
network protocols, compilers and intelligent reasoning 
systems; 

• Methods for analyzingthe complexity of software; and 

• Techniques forsoftware requirements analysis, design 
and testing. 

A similar approach is taken within the otherengineering 
disciplines. Such an approach does notprovide exposure 
to the challenges inherent in a complex system requiring 
knowledge ofmultiple disciplines. Therefore, it doesnot 
allow the students an opportunity to learn the skills 
needed for success in such a project. Furthermore, it can 
give students the false notionthatthey can ignore certain 
aspects of a complex system. For example, a software 
engineer who is unaware of a software performance 
constraint that is derived from the physics of a system 
may develop software that simply ignores such a 



constraint, leading to eventual malfunctioning of the 
system. The faculty involved propose to address this 
serious problem. 

How can one evaluate whether the proposed solution 
solves the problem? First, it must be determined whether 
the solution does indeed address the problem. A good 
solution will address the problem if and only if it offers 
the students an opportunity to develop the skills 
necessary for functioning in multi-disciplinary teams for 
the purpose of engineering complex systems. Second, 
one should consider whetherthe proposed solution could 
be implemented without causing other problems. It 
should not increase the length of time required to 
complete an engineering degree, and the depth and 
breadth provided by current engineering programs 
should not be compromised. The required course 
sequences should change very little. Finally, the issues 
ofcoordinatingand assessing multi-disciplinary teams of 
students should be addressed. 

POTENTIAL SOLUTIONS 

When considering possible solutions to the problem, two 
candidates were identified. One possibility is to teach 
students about the skills developed in each discipline. 
While it is probable that this solution could be 
implemented without causing other problems, it would 
not provide the students with an opportunity to work 
with individuals in other disciplines; consequently, the 
depth of learning would not be very great. A second 
possibility is to have teams of students from various 
disciplines cooperate in an engineering endeavor. This 
would address the problem, since it would allow students 
from diverse disciplines to learn how to cooperate 
synergistically. Would it create additional problems? 

Currently, Mechanical Engineering and Electrical 
Engineering curricula include a “Capstone Senior 
Design” course in w hich the students apply their skills to 
a complex problem in their discipline. Similarly, Com- 
puter Science students take a “Software Engineering” 
course in their final year. The faculty involved propose 
to have students in these courses work on multi- 
disciplinary team projects; this approach would avert all 
of the aforementioned problems, except for the 
coordination and assessment of multi-disciplinary teams. 
To address the issue of coordination within a multi- 
disciplinary team, the faculty involved propose to 
include students from the College of Business in each 
team. This will allow the business students to practice 
skills such as schedule planning and milestone tracking, 
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communication, integration and testing, risk mitigation. 
It would also allow the engineering students to focus on 
the engineering challenges and to benefit from the 
management skills ofothers. It is alsoproposed to utilize 
one TA from each engineering discipline. The 
responsibilities of each TA will include assistance with 
laboratory work, assessment of work and assistan ce with 
the multi-disciplinary curriculum development. 

PRODUCTS 

Since a multi-disciplinary team project addresses the 
problem and can be implemented in a manner that does 
not create additional problems, the faculty involved 
propose to conduct a pilot project during the 2000-2001 
academic year. The specific products that the faculty 
involved will develop include: 

• Course materials for amulti-disciplinary senior design 
course; 

• A set of potential projects for such a course for future 
years; 

• A reusable, extensible platform for multi-disciplinary 
senior design projects; 

• Procedures and policies for multi-disciplinary team 
communication and coordination; 

• A proposal to the National Science Foundation 
Education Directorate; and 

• A scholarly publication for submission to an 
educational conference and/or journal. 

ROBOCUP PROJECT 

One question thatneeds an answeris “What would make 
a good Multi-disciplinary Project?” The problem of 
building a team of cooperating mobile robots is a good 
one. It involves mechanical, electrical and software 
design. It is sufficiently complex to present a challenge 
to students, but is feasible for them to complete in the 
three-quartertime allocated for a Senior Design course. 
Although an initial investment in hardware is requested 
from internal funds (more than equally matched by 
departmental funds), the result would be a reusable 
platform; thus, the cost in subsequent years would 
decrease because the senior design projects could 
involve incremental refinement of the robotic platform. 
Another reason for choosing th is particular project is that 



there is an international competition that students can 
enter with their team of robots. The competition is 
RoboCup — a tournament in which teams of mobile 
robots compete in the game of soccer. The common goal 
of competing in the RoboCup event would help to 
promote cohesiveness and co mmitment within the multi- 
disciplinary team. Thus, the faculty involved request 
funds for representatives of the team to travel to the 
competition. Comp eting in the RoboCup event would be 
a valuable educational experience for the students. 
Another benefit of having the students compete is that it 
would increase the visibility ofthe University. RoboCup 
has been featured on the Discover Channe l’s Scientific 
America Frontiers program. 

MULTI-DISCIPLINARY WORK EFFORTS 

The cooperating mobile robots project offers many 
challengingaspects forMechanical Engineers, Electrical 
Engineers, Software Engineers and Managers. The ME 
students will be responsible for design andintegration of 
mechanical hardware and low-level control. The 
specific tasks are: 

• Defining the specification of dynamic performance; 

• Selecting/sizing the drive-train of the robotesigning/ 
fabricating/assembling the mechanical parts of the 
robot; 

• Designing/fabricating/assembling a kicking 
mechanism; 

• Designing/implementation/testing low-level servo 
control (software/hardware) with EE students; 

• Development of on-line dynamic path planning; 

• Documenting/communicating with EE, CS, MIS to 
integrate the overall system; and 

• Testing/evaluating of the overall system w ith EE, CS, 
MIS. 

The Electrical Engineers will contribute to the team in 
the following areas. 

• Design and implementation of the robot’s on-board 
computing hardware 

• Design and implementation of the robot’s on-board 
system software 
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• Design and implementation of the wireless 
communication system 

• Selection and integration of the robot’s on-board 
sensors 

• Design and implementation of the motor drive and 
control system with ME students 

• Overall system integration with ME, CS, and MIS 
students 

• Evaluation and test of the overall system with CS, MIS 
students, and ME 

The Software Engineers will focus on the following 
aspects of the problem. 

• Design, development and testing of all software 

• Vision and sensing processing 

• Artificiallntelligence (Planning, Strategy analysis, and 
Learning) 

• Distributed Systems 

• Real-Time Computing 

The MIS/Business students will apply the following 
skills. 

• Resource Management 

• Time Management 

• Risk Assessment & Risk Control 

• Unit & System Testing Design 

• Teamwork & Leadership 

• Quality Assurance 

• Software Inspection & Walkthroughs 

• System Planning, Analysis & Design 



IMPACT AND EVALUATION 

The initial project impact will be small in terms of 
number of students benefiting because the faculty 
involved plan a small pilot intentionally. However, 
assuming project success, this innovative multi- 
disciplinary approach to senior design courses could 
impact every senior in the respective disciplines 
(approximately 180 students graduated per year) in the 
future. The project focuses on three-quarter efforts, 
rather than just one quarter. Also, this project can serve 
as a model for the College ofEngineering & Technology 
and potentially many areas within theUniversity that can 
benefit from a multi-disciplinary senior capstone course 
approach. The project impact will be significant in 
improving the planned NSF Education proposal with 
concrete experience and demonstrated the University 
investment. The project will also bring name 
recognition to the University through the RoboCup 
competition. 

The one-year pilot project will be evaluated by 
comparing the multi-disciplinary student teams’ results 
against those of senior students who go through the 
traditional single-discipline three-quarter senior design 
sequence. This will be challenging in that the hardware 
projects will be different; however, the quality, 
robustness, and depth of products can be compared 
between the single- and multi-disciplinary approaches. 
The faculty involved believe thatthe multi-disciplinary 
approach is significantly stronger; however, another 
important evaluation criterion is logistics: did the pilot 
project succeed in terms of large teams, different 
departmental admin istration, and d ifferent backgrounds? 
Part of the result will be a plan for improved multi- 
disciplinary teamwork in undergraduate education. 

SUMMARY OF WORK TO DATE 

The proposed RoboCup team has been holding 
collaborative, multi-disciplinary design meetingsduring 
Winter and Spring quarters, 2000. The faculty involved 
are primarily talking about the proposed multi- 
disciplinary senior design curriculum, but the faculty 
involved are also currently involving undergraduate 
students in preliminary mobile robot designs, sponsored 
by the NSF Research Experiences for Undergraduates 
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(REU) program. The figure below showsa CAD model 
of an early conceptual design for a mobile soccer- 
playing robot. 

FIGURE 1 

EARLY CONCEPTUAL DESIGN 
FOR ROBOCUP MOBILE ROBOT 




During Winter 2000 Dr. Peter Stone addressed the team 
regarding RoboCup via the Stocker Visiting Lecturer 
program. Dr. Stone, employed by AT&T, recently 
graduated from Carnegie-Mellon University (CMU), 
where he has competed successfully in RoboCup since 
its inception in 1997. 

Two teams of Electrical Engineering undergraduates 
competed in an IEEE competition for mobile robots 
collecting steel and brass balls autonomously. Though 
this project had little input from Computer Science and 
Mechanical Engineering, the problem is related to 
mobile robots playing soccer, so the University has 
started building experience in this area. 

Currently Mechanical Engineering is teaching ME 455, 
Mechatronics. The class project is to convert a mobile 
toy car from joystick control to autonomous on-board- 
computer-based control. This project is closely related 
to the RoboCup needs, spanning the software, 
electronics, and mechanical disciplines. This can be 



done better with involvement from the three areas rather 
than just one. 

During the Fall and Winter 2 000 and 2001 the students 
have enrolled in the various senior projects courses and 
have built prototype robot components. During the 
Spring 2001 the students implemented the prototypes 
and during the summer 2001 the competition was held. 
Results will be presented at various conferences in the 
Fall 2001. Multiple papers on various theoretical and 
teaching aspects of this project have been accepted and 
presented at several conferences in various disciplines. 

VARIOUS BUDGET DATA 

The graduate teaching assistants (TA) are crucial for 
project success, working with the Pis, undergraduate 
students, and technicians both on multi-disciplinary 
curriculum developm ent and on practical robot hardware 
design, construction, programming, and control. The 
total requested budget is $24,000, for two teaching 
assistants working for four quarters at $3,000 per 
quarter. The Internal Education Fund is requested to 
provide half this amount and the departments will 
provide matching for an equal amount. The College of 
Engineering & Technology will also provide tuition 
scholarships for these eight quarters of work. 

The hardware budget is intended to purchase required 
components for building the team of RoboCup robots. 
These components include motors, motor drive circuitry, 
PC boards, wireless communication equipment, sensors, 
and video cam eras w ith fram e grabb ers, wh eels, and gear 
transmissions. The early designs indicate a cost of 
approximately $3,500 per robot for six robots, plus a 
one-time cost of $1,000 for communication hardware, 
which is used for all six robots, for a total of $22,000 in 
hardware components. The Internal Education Fund is 
requested to provide $12,000 for hardware, to be 
partially matched with $1,000 from the College of 
Engineering & Technology, $ 1 ,000 from the Pace Grants 
Consortium, $6,000 from the Electrical Engineering and 
Computer Science department, and $2,000 from the 
Mechanical Engineering department. 

A total of $500 is requested from the Internal Education 
Fund to provide required supplies for supporting robot 
construction. Though not shown formally as matching, 
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the College of Engineering & Technology and 
departments will contribute to supplies via existing stock 
and purchases. 

The travel budget is intended to partially support: 1) 
sending a portion of the RoboCup team (Pis, 
undergraduate students, andTAs) to Seattle during the 
summer of 200 1 to represent the University for the first 
time at the international, high-profile RoboCup 
competition; and 2) presenting project results (multi- 
disciplinary senior design projects) at an educational 
conference. The Internal Education Fund is requestedto 
provide $5,000 for travel, to be matched equally by the 
College of Engineering & Technology ($ 3,000), and the 
departments ($2,000). 

Finally, technician labor is required to assist students, 
TAs, and the Pis in producing polished, professional 
robots worthy of representing the University, built for 
reliability. The College of Engineering & Technology 
will provide the sole support, $1,000 for each of two 
technicians (machinist and hardware/software 
technicians). This will provide a total of 100 technician 
hours at $20 per hour. 



CONCLUSION 

This course/project offers theopportunity fora variety of 
disciplines to work together in a learning community to 
accomplish a challenging task. It provides a multi- 
disciplinary learning environment that does not exist on 
very many campuses. It encourages students to think 
about issues related not only to their own major, but 
more global issues that are extremely relevant to real 
world design situations. The project has the potential to 
enhance and extend the reputation o f the University as a 
leaderinequippingMechanical and Electrical Engineers, 
Computer Scientists, and Business students for the fast- 
paced, dynamic, team -orientedcareers that are becoming 
the norm in industry. The proposed project for this 
activity, the international RoboCup competition, 
promises to provide the University high-visibility, 
positive exposure to help recruit quality undergraduate 
students. This model could be adopted by any university 
with engineering, computer science and information 
systems departments. 
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CULTURAL AND GENDER ISSUES IN THE 
COMPUTER AND INFORMATION 
TECHNOLOGY CURRICULUM 



Panel 

Mary J. Granger 
George Washington University 

Joyce Currie Little 
Towson University 

Susan K. Lippert 
Drexel University 



The members of the panel participated in a working group at the 4 th Annual SIGCSE/SIGCUE ITiCSE (Integrating 
Technology into Computer Science Education) addressing this issue in a final paper that will be published in ACM 
SIGCSE 2001. They also contributed to previous working groups concentrating on professionalism and ethical issues 
in the Computer and Information Technology Curriculum. 



ABSTRACT 

Industry leaders and educators in Computer and Information Technology (C1T) have expressed a needfor graduates 
to have a background in professional, societal, and ethical concerns as well as a strong technical capability (Huff 
and Martin, 1995). Some educators have gone so far as to include cultural awareness: “The cultural dimensions of 
information technology can no longer be ignored, with the expansion of the global economy, global markets and 
global communication enabled by information technology ” (Hasan and Ditsa, 1998, p. 5). The rationale for 
supporting and enhancing instruction in culturalissues for CIT workers comes from the growing globalization of the 
world in communication, the increase of trans-national organizational mergers and partnerships, the merging of 
various populations within national boundaries, the increasing traffic of individuals to different countries around the 
world, and the severe shortage of information techno logy personnel throughout the world. This panel will provide 
material to support the inclusion of cultural issues within the CIT curriculum. The topics identified, which include 
diversity and multiculturalism, organizational cultures, professional cultures, socio-economic issues, and gender 
issues, form a foundation body of knowledge that, once learned, can improve and enhance the work of the information 
technology professional. Some exercises are provided that can be incorporated into existing CIT courses across a 
wide variety of programs, nations, and cultures. As the internationalization of education continues, more exercises 
and examples will surely arise from the CIT community. 
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PANEL PROPOSAL 

Hofstede (1991 , p. 5) defines culture as “the collective 
programming of the mind which distinguishes the 
members of one group or category of people from 
another. “More simply, culture is shared values of a 
particular group of people (Erez & Early, 1993) and 
culture reflects the core values and beliefsof individuals, 
which are formed during childhood and reinforced 
throughout life (Shore & V ankatachalam , 1996). This 
implies that culture is all pervasive and has a strong 
influenceon all ourundertakings (Hasan & Ditsa, 1997, 
p. 6). How we view and design com puter systems is also 
defined by our culture, despite the tendency of Western 
scientists to see themselves as being objective and 
transcultural (Harding, 1998). 

Students in today’s university programs will be the CIT 
professionals of tomorrow. Industry and workplace 
profession als realize that skills beyond the techn ical are 
vital to the success of the student. Instructors have a 
responsibility to the students of today to prepare them 
for the world of tomorrow in both the technical arena 
and in the cultural one. In many settings, instructors are 
not prepared to teach such topics as these without 
substantial assistance. Tutorials and workshops are 
needed to bring teachers together to discuss cultural 
matters, to allow the m to learn how they are im portant to 
the workplace, and to help them learn howto teach them. 

Cultural issues are becoming more important as the 
world moves toward increased communication and 
globalization. Many corporate mergers occur across 
national boundaries, requiring personnel to be more 
aware of national and ethnic differences. Also, those 
students from more economically successful countries 
have better access to the technology, and, in addition, 
have more opportunity to attend classes and gain 
knowledge. Recent reports from around the world 
conclude that this economic gap has caused a “digital 
divide that will continue to worsen withoutgovernment 
support and intervention (Smythe, 2000; Independent 
Television Service, 2000). The shortage of information 
technology workers in the United States is well 
documented (Freeman and Aspray, 1999; Anderson, 
1997). The organizational world has addressed the 
shortage by delaying retirement, increasing productivity 
expectationsand encouraging governments to liberalize 
immigrations policies (CPS T, 2000). Another emerging 
trend among corporate giants is “outsourcing of 
informationtechnology work, as many companies move 



work to other countries where personnel are available, 
often at less cost (Van den Berg and Mantelaers, 1999). 

The occupational demographics in the United States 
show that most information technology personnel are 
male Caucasians, with the representation of ethnic 
minority workers very low. A similar disproportionality 
is evident in Australia andNew Zealand with indigenous 
and Pacific Island peoples underrepresented. 
Additionally, in many countries there are drastically 
fewerwomen in the CIT workforce thanmen (Harrelson, 
1998; Camp, 1997). A great many support groups for 
women in IT have arisen in the United States to study 
and promote the proportion of women in computing. 
Even though much progress has been made by these 
efforts, there is still much more to be accomplished 
(Dwight, 2000). The shortage of information technology 
workers overall is exacerbated by the lack of appropriate 
numbers from minority ethnic grouppopulations and the 
lack of appropriate numbers of women (Bajcsy and 
Reynolds, 2000). 

Studies of graduates in the United States reveal what is 
now called the “pipeline effect, which refers to the 
decreasingnumbers of women completingCIT programs 
in universities (Camp, 1997). Graduates in infomiation 
technology related programs, whether computer science 
or other types, include small numbers of women and 
minorities. The same trend is visible in many other 
industrialized countries (01nes,1999; Vosseberg, 1999; 
Selby et al, 1998). Causes of the under-representation of 
women students in CIT inc lude “the structural conditions 
of the teaching environments, a lack of women lecturers, 
lackof knowledge about career prospects among girls, as 
well as the image of computer science and infomiation 
technology and computing as a male domain, and a 
perceived lack of confidence amongst women students 
despite their obvious abilities and successes. (Salminen- 
Karlsson, 1999; Selby et al, 1998). 

Some of the cultural areas to be addressed in the panel 
are: 

Multiculturalism refers to the diversity of our world 
population, differencesofpersons from multiple ethnic 
groups and multiple national origins, andthe ongoing 
tendency toward globalization of our world society; 

Organizational cultures refers to differences invarious 
organizations and how they function, as well 
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as differences between public government and non- 
profit organizations and private industry. Policies 
establishedfor the organization, such as privacy, equal 
opportun ity, imm igration for w orkforce e nhancem ent, 
electronic mail, copyrightand patent law policies, etc., 
are to be included in this category; 

Professional cultures refers to the broad field of 
computing and information technology as an industry. 
This category includes topics such as workforce 
supply/demand studies, personnel research, acceptable 
professional standards and practices, codes of ethics, 
personality profiles of the workforce, characteristicsof 
workers, acceptable educational practices, certification 
and licensing requirements, and need for continued 
updating and training of professional workers; 

Socio-economic issues refers to equal opportunity to 
education, equity of access to technology usage, and 
the growing concern that technology is further 
dividing the have’s from the have-not’s; and 

Gender issues refers to differences between men and 
women in behavio r and in their role in the information 
technology field. It can also refer to actions related to 
the treatment of, and the behavior toward, persons of 
different gender. 

Many organizations in the computer and information 
technology arena are moving toward increased 
globalization and increased work in multinational 
settings. With the worldwide shortage of computer and 
information technology personnel and the worldwide 
development of computer systems for multinational 
audiencesusingmulticulturalpersonnel, there is an even 
stronger need for understanding cultural differences. 

This panel is meant to raise awareness among the 
academic community that cultural issues are important 
issues in the CIT curricula. It also provides exercises 
that can be incorporated into courses within a variety of 
CIT programs and courses. 
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IN THE MANAGEMENT OF IT COURSES 
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ABSTRACT 

Many AACSB accredited schools require undergraduates MIS majors to take a course in the management of 
information technology. Over half of these schools utilize case studies in the teaching of this course. We believe that 
case studies are an important vehicle for teaching crucial IT management issues, particularly in providing students 
with a real-world example of organizational issues. We believe that case studies are best taught in an active, 
collaborative environm ent. Based upon our understanding of collaborative learning and collaborative teaching, we 
propose a procedure for enhancing the effectiveness of this active learning methodology. 



COLLABORATIVE LEARNING 

Collaborative learning is defined as a learning process 
emphasizing group or cooperative efforts among faculty 
and students, stressing active participation and 
interaction on the part of both students and instructors 
(Brufee, 1984). Collaborative learning has long been 
stressed as an effective teaching methodology by 
theorists (Vygotsky, 1978 ). A review of the literature of 
peer/collaborative learning can be found in McKeachie 
(1999). 

The importance of collaborative learning extends to the 
business environment, particularly in the use of teams to 
accomplish business tasks. The effectiveness of using 
teams to accomplish information systems tasks in the 
business environm ent is well recognized (El-Shinnawy 
and Vinze, 1998; Janz, 1999). Surveys of employers 
indicate that teamwork skills are among the most 
important when evaluating IS graduates for entry-level 
positions. Employers have rated teamwork skills as 
more important than systems analysis and design, 
database, or programming skills (VanSlyke, Kittnerand 
Cheney, 1998). 



Collaborative learning is recognized as an effective 
teaching methodology in MIS programs in the United 
States. Through collaborative learning, students learn to 
take advantage of each team member’s expertise and to 
experience first-hand the problems of coordinating a 
team effort (Goyal, 1995/1996). Studies have shown 
that collaborative learning leads to a higher degree of 
satisfaction with the learning process, to a greater 
motivation to learn, and to better performance (Flynn, 
1992). 

Aram and Noble ( 1 999) argue that the traditional lecture 
approach does not adequately prepare students to 
understand and cope with the levels of ambiguity and 
uncertainty they will inevitably face when assuming 
entry level positions. Collaborative learning can be 
utilized in a number of class settings, and it is 
particularly appropriate for system development projects. 
Collaborative learning can also be effectively used for 
research projects and simulations. However, it is our 
intentionto focus on how teac hers can utilize techniques 
of collaborative learning in the teaching of case studies 
in the Management of IT course. 
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COLLABORATIVE TEACHING 

Instructors in a collaborative teaching environment can 
also realize some of the same advantages enjoyed in a 
collaborative learning environment. In particular, 
instructors can benefit from synergistic effects promoted 
by team dynamics. In addition, collaborative teaching 
can lead to a quicker development of best teaching 
practices. 

Brabston, et al. (1999) proposed three models for 
collaborative team teaching. 

1. The interactive model. Two or more instructors in 
front of the class at any one time. 

2. The rotationalmodel. Each member of the teaching 
team teaches in only that part of the course related to 
his or her area ofexpertise. 

3. The participant-observer model. Each team member 
alternately takes the lead in teaching. The other 
team member primarily observes but also actively 
participates when appropriate. 

We will later present a fourth model for collaborative 
team teaching. 

CASE STUDIES 

Case studies are an important tool for teaching MIS 
concepts. Romm and Pliskin (2000) cite a study by Lee, 
Trauth and Farwell (1995) showing four major clusters 
of knowledge/skills required of MIS personnel in the 
upcoming decades. 

1. Technical Specialties Knowledge/Skills: including 
operating systems, programming languages, 
database management systems, networks, 
telecommunications, etc. 

2. Technology Management Knowledge/Skills: 
including issues such as where and how to deploy 
information technologies effectively for meeting 
strategic business objectives. 

3. Business Functional Knowledge/SkiUs: including 
how to re-engineer business processes before the 
adoption of a new information system to produce 
maximum benefit from the system. 



4. Interpersonal and Management Knowledge/Skills: 
which relate to the “boundary-spanning role of IS 
personnel. This role requires IS professionals to 
master interpersonal skills such as selling, 
negotiating, leading, and counseling. 

Romm and Pliskin (2000) note that of these four skills, 
three of them technology management, business 
knowledge, and interpersonal skills are not the 
traditional“hard skills associated with an IS education, 
but rather can be classified as “soft skills because they 
emphasize an understanding and ability to work with 
people rather than machines. Romm and Wong ( 1 997) 
persuasively argue that the best way of teaching these 
soft skills is through the use of case studies. 

Case studies are routinely used in a number of MIS 
courses. In a content analysis of34 electronic commerce 
course syllabi, Sendall (1 999)found that 44 percentof e- 
commerce classes were incorporatingcase studies as part 
of the curriculum . 

Our focus is on the IT management course at the 
undergraduate level. We believe that this course is 
critically important to the MIS major, and we share the 
surprise of O’Hara and Stephens (1999), who found that 
this course is not universally required at AACSB- 
accredited schools. In their study, O’Hara and Stephens 
content analyzed 39 undergraduate syllabi of the IT 
management course. They found that the most common 
assessment method of students in this course were 
exams, quizzes, case study analyses, research papers or 
topic studies, computer-based projects, reports, and 
assignments. Of the 39 courses, only 5 1 percent utilized 
case studies. Further, case analyses accounted for only 
16 percent of the grade, on average. 

TEACHING CASE STUDIES 
THROUGH COLLABORATIVE LEARNING 

Case studies can be taught with many different 
methodologies. Romm and Mahler (1 99 1) describe five 
methodologies. 

1. Individual processing Students prepare for cases as 
individuals. 

2. Chronological group discussion . Each case is 
presented via a team (and instructor) with the team 
intact throughout the interactive discussion. 
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3. Simultaneous group discussion . Each case is first 
discussedin sub-units, which later recombine as one 
large group. 

4. Chronological group dramatization . Cases are 
dramatized with the all students serving either as 
actors or audience. 

5. Simultaneous group dramatization . Students first 
break into sub-units which later recombine for case 
dramatization. 

Each of these methodologies has its virtues, and each 
involves a certain amount of active learning. Annette 
Jones (2000) summarizes the argument for active 
learning. 

Active learning is based on the assumption that 
learning is by nature an active undertaking, and 
that different people learn in different ways 
(Meyers & Jones, 1993); it presumes that 
students learn best by doing. Active learning 
provides opportunities for students to talk and 
listen, read, write and reflect on course content 
through problem-solving exercises, small group 
discussions, simulations, case studies and other 
activities. Biggs(1999) also suggests that active 
engagement in the learning process encourages 
the less academic student to employ high-level 
engagement techniques such as theorization, 
reflection, application, whicharemore naturally 
adopted by the more academic student even if 
the teaching method is more passive. 

We believe that one of the greatest strengths of teaching 
cases is the flexibility which they provide the instructor. 
A teaching case allows an instructorto choose the level 
of depth for discussion of a topic, as well as which 
topics, theories, and practices are discussed. While 
many teachers have developed their own pedagogical 
methods for teaching cases, there is no generally 
acceptedprescription for one “right way to teach cases. 

As we continue to sharpen our teaching skills, several 
questions occur to those of us that teach cases in class. 
These questions generally center on inquiry as to 
whether our approach is the most appropriate. 
Generally, the purpose of case instruction is to provide 
a real-world example of the issues that organizations 
must face. Such exposure allows students the 
opportunity to identify issues and problems faced by a 
firm, to see vague, conflicting and often ill-structured 



business scenarios, to evaluate decisions made by the 
principles, to relate theory and concepts to a specific 
instance, and/or to make recommendations about what 
should be done based upon thestudent’s own knowledge 
of the subject matter. As such, it is always ourhope that 
the material will “come alive for the students, 
generating high interest because of the fact that the 
issues are real and the companies are struggling to deal 
with them. 

Most teachers who have been teach ing cases fo r a while 
have developed his or her own particular method for 
conductingdiscussionof a case in class. We next sketch 
out the two methodologies we have most recently 
followed. 

CASE STUDIES IN THE IT MANAGEMENT 
COURSE AT OUR UNIVERSITY 

Each of the two authors of this study teach the IT 
Management course. For the past three years, one of us 
teaches the course in the fall semester, and the other in 
the spring. Both of us are proponents of active, 
collaborative learning in the teaching of cases. Both of 
us are interested in improving the effectiveness of our 
teaching. We have in the past informally discussed the 
teaching of cases in our respective classes, and have 
determined that a more formal approach to improving 
our teaching is in order. 

As a result of our informal discussions we have 
developed a case study evaluation instrument. It was 
administered for the first time in Spring 2001, and our 
intention is to administerthe same instrument in the fall 
class. The following two sections describe the case 
teaching methodologies employed by each instructor in 
his respective section. 

The Fall 2000 IT Management Class 

I assign a short, end- of chapter cases at least one week in 
advance. I ask the students to read the case and answer 
the questions included in the text after the case. The 
students’ written (word processed) answers are turned in 
to me after the case is discussed in class. I make it very 
clear that their work is not graded in terms or “right or 
“wrong ; instead, I simply look at each paper to 
determine whether a thoughtful and justifiable response 
has been formulated. Once the cases have been turned 
in, I grade each student’s work by assigning a check 
mark (S) or minus (-) indicating whether I have deemed 
their work to be sufficient. Insufficient answers are 
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relatively rare. Those receiving a check mark are given 
credit for all of the points for the assignment, while those 
who receive a minus receive no credit. 

The written answers to the case questions serve as 
reference for the students as we discuss the case in class. 
Because of the availability of some written guidance, 
many of the students seem more at ease when they are 
called upon to contribute to the discussion. Also, I have 
noticed that studentsseem to be more prone to add to the 
discussion voluntarily when they have a well-formulated 
response in writing at their disposal. 

I purposefully don’t read the questions accompanying 
the case before the class discussion. Instead, I work up 
my own set of questions. My reasoning for this 
approach is that if I read the questions prepared for the 
case, I may actually constrain my own thoughts about 
the issues. I prefer to lead the discussion on what I feel 
is most important to emphasize. Only after I have 
exhausted my own list of questions for the class will I 
ask for responses to the given case questions if, in fact, 
we have not already covered the question in our 
discussion. 

When I have a classroom thatallows for rearranging the 
student seating, I ask the students to move their chairs 
into a circle. I also sit in the circle. This arrangement 
seems to improve the informality of the setting and is 
very conducive to group discussion. 

Student comments, both formally through instructor 
evaluation reports and informally through discussion 
with individuals, have been very positive about the value 
of teaching cases in my classes. The anecdotalevidence 
for the success of this approach is strong but indirect. 

The Spring 2001 IT Management Class 

Inspired by a workshop hosted by Larry K. Michaelsen 
(see Michaelsen, 1997-1998) priorto the beginning of 
the semester and disappointed by the negative feedback 
I received from my spring 2000 class, I made several 
sweeping changes to my case study methodology. In 
spring2001, 1 divided my class of 22students into seven 
teams. I consciously used principles of demographic 
diversity as advocated by Trimmer, Van Slyke and 
Cheney [1999] in comprising the teams. I also ensured 
that there would be at least one “high-performing 
studenton each team. Theteams endured throughoutthe 
semester. Although Michaelsen strongly advocates 
giving students all the class time they need to operate in 



groups, I purposefully composed teams whose members 
shared some free time at some point during the week. 
Further, I encouraged teams to conduct e-meetings to 
discuss case questions and to prepare the final report. 

Five case studies were assigned at the beginning of the 
semester, with case discussion to begin on the fifth week 
of the semester. The cases came from Turban (1999). 
Text questions for each of the five cases were 
supplemented with my own questions. 

On each case studies day, the period began with all 
students taking a short multiple-choice quiz on the 
details of the case. As advocated by Michaelsen in his 
workshop, students then immediately grouped together 
in their teams to discuss the quiz and to retake it, this 
time as a group quiz. The quizzes served to motivate 
students to read the case carefully. 

I then led the discussion of the case questions in a 
question-answer format. Students frequently enlivened 
discussion with vigorous debate, as opposing points of 
view were enthusiastically presented. Invariably, discus- 
sion would fill the remainder of the class period, and the 
following class period was dedicated to tying up the 
loose ends of the case. 

One week after the case discussion concluded, each team 
submitted a written analysis of the case. At the end of 
the semester, all students rated the relative contributions 
of their teamm ates. Students were forced to give at least 
one teammate more points than the rest, a practice 
advocated by Michaelsen. Altogether, the case work 
(quizzes and reports) was worth 25 percent of the course 
grade. Forcing students to allocate points unevenly led 
to several students having their course grade elevated or 
demoted a level. 

Instrument Development and Administration 

We developed a 12-item questionnaire administered in 
the spring semester in a collaborative fashion. The 
instrument itself (see Appendix A) is designed primarily 
as an exploratory tool to assess the effectiveness of our 
approach to teaching case studies. We believe a basic 
value of the case approachis in the teaching of soft skills 
(Romm & Pliskin, 2000), which led to the development 
of questions two and six. We believe that the case 
approach is an excellent vehicle for teaching the 
Bloom’s (1965) higher levels of learning, hence 
questionsfive, seven, eleven, andtwelve. Because many 
of the changes in appro ach were inspired by Michaelsen 
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(1997*1998), we developed questions four, eight, and 
nine. We developed question three to assess students’ 
perceptions of cases as agents of active learning, as 
suggested by Horgan (1999). Both instructors felt that 
cases were an important tool for teaching key issues, 
resulting in question one. Finally, we wanted to learn 
whether students preferred our new approach to the more 
traditional case approach, which accounts for question 
ten. The instrument concludes with two open-ended 
questions to explore issues not sampled by the first 
twelve questions. 

At semester’s end, students anonymously evaluated the 
case studies on a five-point scale. The results of this 
survey are shown in Table 1. Questions have been 



sorted from their original arrangement, to an order 
showing statements most strongly agreed with first. 

The results of the survey indicate a widespread 
satisfaction with the approach to case studies. Our 
perception is that students like this case methodology 
much more than they did in the spring 2000 class. 
Students seem to be particularly satisfied with cases as 
tools for making abstract MIS principles concrete. Not 
surprisingly, the question which received the least 
support concernedthe forcing ofuneven ratings which 
led to some students receiving lower grades for the 
course than they otherw ise would have. However, only 
two of the twenty-two students either disagreed or 
strongly disagreed with question nine. 



TABLE 1 

RESULTS OF CASE STUDIES QUESTIONNAIRE 



Question 

Number 


Question 


Mean 


7. 


The cases provide students with a good means of applying information systems principles to 
real world situations. 


1.36 


1 . 


The cases brought out important points about managing in formation systems, such as the role 
of IS in a global economy, the potential of e-commerce, the role of IT in strategic planning, 
IT ethics, etc. 


1.41 


11 . 


The cases provide students with a good opportunity to synthesize; that is, identifying 
potential solutions to a case problem and choosing the most appropriate solution. 


1.45 


12. 


The cases provide students with a good opportunity to exercise evaluation skills, i. e., 
appraising the extent to which particulars are accurate, effective, economic, or satisfying. 


1.45 


4. 


Having both an individual quiz and a group quiz is a good idea. 


1.55 


2. 


The cases are a good way of teaching “soft skills ; for example, interpersonal skills and 
management skills. 


1.64 


3. 


Cases increase the likelihood of student participation in class discussion. 


1.64 


6. 


Because much of the case work involved team work, the cases served as a good vehicle for 
applying principles of team management. 


1.68 


5. 


Writing the case report aided in understanding the case principles. 


1.73 


10. 


I prefer the approach to cases used this semester to the traditional case approach, i.e., when 
student teams are assigned the responsibility of presenting a particular case. 


1.73 


8. 


Requiring students to assess relative contributions of teammates is a good way to motivate 
individual efforts. 


1.77 


9. 


Requiring students to rate at least one teammate’s contribution as better than average is a 
good idea. 


2.41 
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Even the response to question ten was gratifying. This 
question, which ranked tenth in order of agreement, 
asked students to compare the spring2001 approach with 
the traditional approach. Although three students rated 
the question neutral, not one of the twenty-two either 
disagreed or strongly disagreed with the statement. 

The results of this questionnaire must be interpreted with 
caution. Apparent student satisfaction with the 2001 
methodology does not ensure that the methodology was 
more effective. There could have been contaminating 
factors the students may have found that pioneering a 
new methodology was a positive experience. Or the 
instructor may have done a more effective job for all 
aspects of the course pulling the case study results up 
by the bootstraps, as it were. Nevertheless, theevidence 
does point to the 2001 methodology being more 
effective. 

In addition to the twelve questions above, students were 
also asked to respondto these two open endedquestions. 

1. What is the one thing you liked best about our 
approach to cases this semester? 

2. If you could change one thingabout our approach to 
cases this semester, what would it be? How would 
you change it? 

As might be expected, a number of students rephrased 
one or more of the twelve statements to indicate the 
greatest strength. As proponents of active learning, we 
are pleased to note that six students indicated that active 
class discussions were the greatest strength of this 
approach to case studies. 

Even responses to the “greatest weakness question 
tended to be positive. In fact, five studentsindicated that 
there was no weakness with the 2001 methodology. 
Typical responses to this question included: 

More than five cases should be used (2) 

More in-class time should be allocated to cases (2) 

Class should be 75-minutes rather than 50 minutes 

Complaints tended to be about the cases themselves 
rather than to the methodology. 



• Cases should be more current (2) 

Cases should be more detailed 

LESSONS LEARNED 

While Brabston, et al. (1999) proposed th e three m odels 
for collaborative team teaching as described earlier in 
this paper, we would like to propose a fourth, less radical 
model. We suggest a more formal approach to 
procedures that good instructors are already doing 
informally. We propose that when two or more 
instructors are assigned responsibility for the teaching of 
a particular class using case studies as a teaching tool, 
that these instructors routinely perform the following 
steps. 

1. Compile a list of generally agreed upon desired 
outcomes from teaching the cases. 

2. Construct a questionnaire designed to evaluate the 
processes used to achieve the outcomes. 

3. Administer the questionnaire at the conclusion of 
each semester. 

4. Meet to discuss questionnaire results; identify 
methodologies that best meet desired outcomes. 

5. Incorporate appropriate methodologies in future 
classes. 

We believe that, while no “one-size-fits-all when it 
comes to teaching methodologies, this process can only 
result in more effective teaching. A structured approach 
to evaluating methodologies from multiple instructors 
teaching a common course, as we have begun to do, 
should lead to a fine-tuning of individual teaching 
performance. 

The case study format presents an excellent opportunity 
for instructors to collaborate in the determination of 
which methods and desired outcomes are most 
appropriate for a course. In this paper, we have 
presented a combination of anecdotal and empirical 
evidence of the benefits instructors (and, ultimately our 
students) can derive from the above model. 
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We believe thisprocess can only result in more effective 
teaching. We believe that there is no one-size-fits-all 
teaching methodology, but we do believe a structured 
approach to evaluating our methodologies, as we have 
begun to do, should lead to a fine-tuning of individual 
teaching performance. 

Particularly given the A ACSB ’s emph asis on assessment 
(Williams and Price, 2000), we believe that ourmodel of 
collaborative teaching should serve as an excellent 
springboard for impro ving instruction. The responses to 
questionnaire items are valuable, butthe discussionthey 
provoke is invaluable. We do not believe it is enough to 
simply divide students into groups to discuss case 
studies. The devil is in the details . 

For example, if many students indicate that “lively 
discussion is a great strength of one instructor’s 
approach, the positive response is not necessarily due to 
the content of the questions. Discussion of this item may 
reveal that lively student participation is a product ofthe 
instructor’s serving more as a facilitatorthan as a leader. 
Discussion of these and other questions lead to 
synergistic improvements. Good teaching is certainly as 
much art as science, but it is an art enhanced by our 
collaborative procedure. 
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APPENDIX A 
CASE QUESTIONNAIRE 



Name: 

Use the scantron to answer the first twelve questions below. Do not put your nam e on the scan tron. You must write 
your name above in order to receive the ten points extra credit. 



A: strongly agree B: agree C: neutral D. disagree E. strongly disagree 

1 . The cases brought out important points about managing information systems, such as the role of IS in a global 
economy, the potential of e-commerce, the role of IT in strategic planning, IT ethics, etc. 

2. The cases are a good way of teaching “soft skills ; for example, interpersonal skills and management skills. 

3. Cases increase the likelihood of student participation in class discussion. 

4. Having both an individual quiz and a group quiz is a good idea. 

5. Writing the case report aided in understanding the case principles. 
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6. Because much of the case work involved team work, the cases served as a good vehicle for applying principles 
of team m anagem ent. 

7. The cases provide students with a good means of apply ing information systems principles to real world situations. 

8. Requiring students to assess relative contributions of teammates is a good way to motivate individual efforts. 

9. Requiring students to rate at least one teammate's contribution as better than average is a good idea. 

10. I prefer the approach to cases used this semester to the traditional case approach, i.e., when student teams are 
assigned the responsibility of presenting a particular case. 

1 1 . The cases provide students with a good opportunity to synthesize; that is, identifying potential solutions to a case 
problem and choosing the most appropriate solution. 

12. The cases provide students with a good opportunity to exercise evaluation skills, i. e., appraising the extent to 
which particulars are accurate, effective, economic, or satisfying. 

13. What is the one thing you liked best about our approach to cases this semester? 

14. If you could change one thing about our approach to cases this semester, what would it be? How would you 
change it? 
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ABSTRACT 

The case study is one of the most widely used tools in many disciplines including Information Management . At the 
same time the new Internet revolution has promoted online distance learning courses as never before. Many online 
courses are also incorporating the benefits of case studies into their course content. Case studies are now available 
online and case discussions can be conducted using asynchronous and synchronous online discussion tools. The 
online case study, or cybercase is an innovative pedagogical tool, however, more research is needed in regards to 
the issues surrounding the implementation of online case studies. The main objective of this paper is to analyze the 
use of online discussion methods for case study discussion. It discusses background issues regarding online cases, 
and themetho d for conducting an o nline case d iscussion. In particular, it inv estigates th e idea that online case studies 
can be powerful learning tools in a distance learning environment. This study compares and contrasts traditional case 
discussionsversus online case discussions. So me of the im portant questions explored in the present research are What 
is the type of discussion that takes place in each setting? What is the level of discussion participation in an online case 
analysis? Is the quality of the discussions in online cases comparable to traditional discussions? What are the major 
similarities and differences in interaction and learning between an online case study discussion and traditional case 
discussion? and What is the most adequate instructor's role in the online case discussion environment? 



INTRODUCTION 

The case study is one of the most widely used tools in 
many disciplines including Information Management. 
The case study allows students to apply theoretical 
concepts to real world actual situations. Cases are an 
important component of the curriculum at important 
business schools (Tracy and Waldfogel 1997) such as 
Harvard Business School (Schmotter2 000, Chow 1992). 
At the same time the new Internet revolution has 
promoted online distance learning courses as never 
before. Many online courses are also incorporating the 
benefits of case studies into their course content. Case 
studiesare now available online and case discussions can 
be conducted using asynchronous and synchronous 
online discussion tools. The online case study, or 
cybercase (Niederhauser 1999) is an innovative 
pedagogical tool, however, more research is needed in 



regards to the issues surrounding the implementation of 
online case studies. 

The main objective of this paper is to analyze the use of 
online discussion methods for case study discussion. It 
discussesbackground issues regarding online cases, and 
the method for conducting an online case discussion. In 
particular, it investigates the idea that online case studies 
can be powerful learning tools in a distance learning 
environment. This study compares and contrasts 
traditional case discussions versus onlipe case 
discussions. Finding a way to explain differences 
between electronic discussion and oral discussion has 
not been simple and many studies have compared 
multiple measures across both environments 
(LaGrandeur 1997). Some of the important questions 
explored in the present research are: 
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What is the type of discussion that takes place in each 
setting? 

What is the level of discussion participation in an 
online case analysis? 

Is the quality of the discussions in online cases 
comparable to traditional discussions? 

What are the major similarities and differences in 
interaction and learning between an online case study 
discussion and traditional case discussion? 

What is the most adequate instructor’s role in the 
online case discussion environment? 

In addition, this study looks at how the method used in 
online discussion can affect the learning outcomes for 
students involved in an online distance learning 
environment. 

THE CASE STUDY METHOD 

The case study is a means of bringing in greater realism 
into the classroom (Lapierre 1996). Cases can be written 
by academics (Summers et al. 1990) or practitioners 
(Kellog 1985). Most cases include a thesis and 
argumentation but not always a conclusion (Gini 1985). 
The instructor’s primary role is to prompt when 
necessary to ensure that critical issues are not 
overlooked. The instructor can also summarize the 
discussion and draw out the useful lessons and 
observations (Ham mond et al. 1999). 

Advantages of the case study include: 

• Vicarious learning. Participants learn as critics or 
observers too. 

The lively interchange of ideas and opinions allows a 
much greater variety of interpretations of the case 
(Szpiro and Neufeld 1999). People can experience 
immediate feedback and reflect on their own 
perceptions (Niemyer 1995). 

Motivation. Participants are more likely to remember 
concepts used to solve real-life problems and being 
able to apply concepts rather than just learning 
textbook theories (Andrews and Noel 1986). The 
whole phenomenon of case-method teachingmotivates 
instructors as well (McCraw 199 9). 



Transfer. Cases give students experience that can be 
applied to subsequent cases, and on the job situations 
(Standridge 2000, Dorn 1999). 

Active participation. Case studies require less lecturing 
and more active learning (Standridge 2 000). 

Cases yield generalizations and help students take 
ownership of knowledge (Robyn 1998). 

Cases promote the development of critical thinking 
skills (Capella and Robin 1986). 

However, there are also some disadvantages (Stonham 
1995) that include: 

Cases are highly dependent on the instructional and 
educational characteristics of the instructor. 

Case discussion can be difficult for a large class size 
(Mostert and Sudzina 1996). 

Some students may have difficulties with writing. 

Some physical setting may not be appropriate for the 
discussion. 

Level of class preparation and complexity. 

Student inexperience with case analysis. 

A written case does not fully convey the complexity of 
the real situation (Andrews and Noel 198 6). 

Cases are a less efficient way to communicate content 
than other methods (Fulmer 1 992). 

IMPLEMENTING AN ONLINE DISCUSSION 

There are two major options for implementing an on-line 
case study discussion: synchronous and asynchronous. 
Several studies (Davidson et al. 2000) haveanalyzed the 
quality of online interactions, both in chat rooms 
(synchronous mode) and bulletin boards (asynchronous 
mode). Chat room discussion allows for immediate 
feedback, while bulletin board discussions encourage 
students to provide thoughtful responses to posted 
questions. In this study, the bulletin board discussion 
mode was used because participants can elaborate on 
their answers and provide more insightful thoughts, 
however, future studies should evaluate the use of chat 
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rooms for case study discussions. The implementation of 
the online case discussion can vary depending on several 
factors including instructions given to students, 
discussion organization, instructor’s participation, 
student participation requirements, and assessment of 
discussion participation (Bailey and Wright 2000). 

The advantages of online discussions over face-to-face 
exchanges have been well documented in the research 
literature. Some of the advantages of online case studies 
include: 

Online case studies can be linked to discussion boards, 
additional content material, online articles, web sites 
for additional information, or other resources. 

Another dimension can be added by using online 
newspaper articles. Newspaper articles reflect the 
business conditions that decision-makers are feeing 
(Schaupp and Lane 1992). Some online cases are even 
mainly based on newspaper or magazine articles that 
are available online. A progression of related news 
stories throughout a period of weeks could show the 
evolution of a major case issue. 

All students have equal opportunity forparticipation. 
An online case discussion format can encourage timid 
students who did not participate in class to participate 
more actively and to express themselves. It is a more 
equitable, less stressful and non-threateningforum for 
discussions, especially for nonassertive personalities 
(Warschauer 1997). 

Chat room discussion can overcome distance 
limitationsby allowing extern al guests to participate in 
the case discussion. Students can prepare their 
questions ahead of time. 

Participants can access the cases anytime over the 
Internet. 

Accessibility, flexibility, group interaction, and 
opportunity for self-direction. 

More available time to elaborate a response. 

Online discussion can provide the opportunity for 
students to engage in thoughtful conversations, which 
may result in deeper understanding and greater 
learninggains (Irvine2000). It encourages all students 
to formulate their thoughts at a deeper level (Bailey 
and Wright 2000). 



Contributed responses become a repository available 
to other students. 

Participants have more time to respond to answers 
because elapse time is much longer. 

The amount of student contributions in comparison to 
instructor contribution is much higher. 

Answers can be more elaborated, edited and carefully 
thought and phrased. Online discussions seem to 
promote rhetorical experimentation on the part of the 
participants (LaGrandeur 1997). 

The same group can maintain simultaneously multiple 
discussions on different topics. 

An onlinecase discussion can be even more interactive 
than a traditional classroom case discussion. 

The online instructor can selectively release content, 
can create multiple discussion groups, can provide 
private or public feedback. 

A stimulus for increased written participation (Kern 
1995). 

An expanded access channel with possibilities for 
creating global learning networks (Cummins 1995). 
Online cases allow the participation of students from 
multiple countries in the same discussion therefore 
promoting a global perspective in education (de Wilde 
1991). Online cases can be used in a multinational 
virtual environment (Zhao 1996). 

Online cases do not have to be always presented in a 
written form (Niemyer 1995). Some online cases are 
multimedia rich and include audio and streaming 
video. Students should be able to re-exposethemselves 
to the case description by re-playing, rewinding or by 
pausing the video case. 

The implementation of online case discussion also 
involves major challenges such as: 

Online discussion can be difficult for a large class size. 

Some students may not be computer literate. 

Hardware, software and other technical problems 
(Niederhauser 1999). 
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Lack of Internet access. 

Limited availability of online cases. 

The available bandwidth for a modem connection 
limits the size of video clips, sound clips, and inline 
images appropriate for an online case. As better 
technologiesforstreaming video becom e available, we 
should see more video cases available online. 

THE CASE DIFFICULTY CUBE 

Cases vary in terms of the volume and complexity of 
elements, some cases are full of facts, descriptions and 
quotes, other give little detailed information (Einsedel 
1995). Some cases explicitly state the issues and 
problems, others do not, instead they provide symptoms 
or clues that suggest underlying problems (Einsedel 
1995). Many authors have provided suggestions on 
effectiveuse of different case study types, with different 
populations, and at differentphases of learning (Romm 
and Mahler 1991). A different approach or objective 
calls for a different case type. The level of difficulty of 
a case can be determined by using the cube framework 
proposed by Erskine et al. (1981). The cube framework 
can be used to classify the different types of cases. The 
cube has three dimensions: analytical, conceptual, and 
presentation. Each of these dimensions can be divided 
into three degrees. 

A. The Analytical Dimension This dimension is 
concerned with the task that the student must 
accomplish in the case. The three degrees of 
analytical difficulty can be summarized as follows: 

Level 1. Both the problem and the implemented 
solution are described. Evaluate the 
appropriateness of the solution. 

Level 2. The problem is defined, a feasible 
solution must be found. 

Level 3. A situation is described, neither the 
problem, nor a solution are defined. 

As the level of analytical difficulty increases, so does the 
required analytical skills. 



B. The Conceptual Dimension This dimension 
describes the complexity of the fundamental 
theoretical concept(s) underlying the case. 

Level 1. The concept, theory or technique is 
simple and straightforward. The concept or 
concepts in the case may be easily grasped by all 
the participants just by reading the case. 

Level 2. The concept or theory is of medimu 
difficulty. It is a combination of concepts, or may 
require further explanation by the instructor. 

Level 3. The concept is difficult, cross-functional 
or complex. Requires the active participation of 
the instructor to explain some of the concepts. 

C. The Presentation Dimension This dimension is 
related to the amount of information that is given 
and how it is presented. 

Level 1. The amount of information required for 
analysis is small. The case is relatively short. The 
information is presented in a clear and straight- 
forward way. There is almost no extraneous 
information in the case. 

Level 2. The case length is average, the 
information is clearly presented and some 
extraneous data is included in the information. 

Level 3. The case contains a large amount of 
information. The information may not be clearly 
presented, or they might be critical information 
missing. The case might include a large amount of 
extraneous information andmay be less organized. 

Figure 1 shows the Case Difficulty Cube (Erskine etal. 
1981) and its three related dimensions. A case-based 
course would begin with cases ranked lower in difficulty 
along all three dimensions and gradually increase the 
level of difficulty along all three dimensions. The total 
number of combinations in the cube is 3 x 3 x 3 = 27. 

One of our objectives is to determine how cases with 
various levels of complexity can be implemented as an 
online case study. Could certain dimension levels be 
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FIGURE 1 

CASE DIFFICULTY CUBE 




more suitable to an online case discussion? The 
implications for implementing an online discussion for 
a simple case might be different from implementing an 
online discussion for a complex and lengthy case. 
Therefore, in this study we implement online case 
discussions for varied case types. We decided to use 
multiple versions of the same case, instead of using 
multiple cases in order to provide some level of 
comparability. The same case is adapted to the particular 
dimension. In the present study we have simplified the 
framework for implementation purposes, and used only 
two levels for every dimension. The number ofdifferent 
case versions in this study is2x2x2 = 8. 

The case used in this study is called “Tradenet: 
Singapore’s Computerizationof International Trade, by 
B. S. Neo (1994). A modified version is available online. 

A. The Analytical Dimension 

Level 1 . Both the problem faced by Singapore and 
the implemented solution are described. Students 
are expectedto evaluate the appropriatenessof the 
Tradenet solution. 

Level 2. The problem faced by Singapore is 
defined, but the information related to the 
implemented solution was eliminated from the 
case. Students must figure out a feasible solution. 



B. The Conceptual Dimension 

Level 1. The focus of the case is on simple 
concepts such as the definition of EDI. Higher 
order concepts were removed from the original 
case version. 

Level 2. The original case deals with more 
complex concepts such as defining a vision, co- 
operation, technology diffusion, partnership, focus 
effort, strategic alignment of information techno- 
logies, integration, implementation, stakeholder 
participation and identification of business needs. 
Specific lessons learned for each concept are 
described. 

C. The Presentation Dimension 

Level 1 . All extraneous information was removed 
from the case. The case was made as short as 
possible without removing any relevant or 
important information. 

Level 2. This version has the original case length. 

Table 1 shows the eight different versions of the case. 
By modifying the case we were able to use the same case 
to represent multiple case types. 

TYPES OF QUESTIONS BASEDON 
BLOOM’S TAXONOMY 

As teachers we tend to ask questions in the “knowledge 
category 80% to 90% of the time. These questions are 
not bad, but using them all the time is. Try to utilize 
higher order level of questions. These questions require 
much more “brain power and a more extensive and 
elaborate answer. Below are the six question categories 
as defined by Bloom (1956). 

Write questions that test skills other than recall. 
Research shows that most tests admin istered by F acuity 
rely too heavily on students’ recall of information 
(Milton, Pollio, and Eison, 1986). Bloom (1956) argues 
thatit is important for tests to measure higher learning as 
well. Fuhrmann and Grasha (1 983, p. 170) have adapted 
Bloom’s taxonomy for test development. Here is a 
condensation oftheir list and the kind of questions you 
should ask to measure each category. 
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TABLE 1 

CASE VERSIONS AND DIMENSIONS 



Case Version 


Analytical Dimension 


Conceptual Dimension 


Presentation Dimension 


1 1 1 


1 (decision made) 


1 (basic concepts) 


1 (short version) 


112 


1 (decision made) 


1 (basic concepts) 


2 (long version) 


121 


1 (decision made) 


2 (medium difficulty concept) 


1 (short version) 


122 


1 (decision made) 


2 (medium difficulty concept) 


2 (long version) 


211 


2 (decision to be made) 


1 (basic concepts) 


1 (short version) 


212 


2 (decision to be made) 


1 (basic concepts) 


2 (long version) 


221 


2 (decision to be made) 


2 (medium difficulty concept) 


1 (short version) 


222 


2 (decision to be made) 


2 (medium difficulty concept) 


2 (long version) 



TABLE 2 

BLOOM’S TAXONOMY 



KNOWLEDGE 

(common terms, facts, principles, procedures) 

remembering 

memorizing 

recognizing 

recalling identification 

recalling information 

who, what, when, where, how ...? 

describe 


COMPREHENSION 

(understanding of facts and principles, interpretation of 

material) 

interpreting 

translating from one medium to another 
describing in one’s own words 
organization and selection of facts and ideas 
retell... 


APPLICATION 


ANALYSIS 


(solving problems, applying concepts and principles to new 


(recognition of unstated assumptions or logical fallacies, 


situations) 


ability to distinguish between facts and inferences) 


problem solving 


subdividing something to show how it is put together 


applying information to produce some result 


finding the underlying structure of a communication 


use of facts, rules and principles 


identifying motives 


how is ... an example of...? 


separation of a whole into component parts 


how is ... related to ...? 


what are the parts or features of...? 


why is ... sign ificant? 


classify ... according to ... 
outline/diagram ... 

how does ... compare/contrast with ...? 
what evidence can you list for ...? 


SYNTHESIS 


EVALUATION 


(integrate learning from different areas or solve problems by 


(judging and assessing) 


creative thinking) 


making value decisions about issues 


creating a unique, original product that may be in verbal form 


resolving controversies or differences of opinion 


or may be a physical object 


development of opinions, judgements or decisions 


combination of ideas to form a new whole 


do you agree that ...? 


what would you predict/infer from ...? 


what do you think about ...? 


what ideas can you add to ...? 


what is the most important .,,? 


how would you create/design a new ...? 


place the following in order of priority ... 


what might happen if you combined .,.? 


how would you decide about .,,? 


what solutions would you suggest for 


what criteria would you use to assess ,,.? 
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TABLE 3 

QUESTIONS TYPES IN EACH CATEGORY 



KNOWLEDGE 


COMPREHEND 


APPLICATION 


ANALYSIS 


SYNTHESIS 


EVALUATION 


Define 

Describe 

Identify 

Label 

List 

Match 

Name 

Outline 

Reproduce 

Select 

State 


Convert 

Defend 

Distinguish 

Estimate 

Explain 

Extend 

Generalize 

Give examples 

Infer 

Predict 

Summarize 


Demonstrate 

Modify 

Operate 

Prepare 

Produce 

Relate 

Show 

Solve 

Use 


Diagram 

Differentiate 

Distinguish 

Illustrate 

Infer 

Point out 

Relate 

Select 

Separate 

Subdivide 


Categorize 

Combine 

Compile 

Devise 

Design 

Explain 

Generate 

Organize 

Plan 

Rearrange 

Reconstruct 

Revise 

Tell 


Appraise 

Compare 

Conclude 

Contrast 

Criticize 

Describe 

Discriminate 

Explain 

Justify 

Interpret 

Suppo rt 



TABLE 4 

QUESTION CATEGORIES IN EACH CASE VERSION 



Case Version 


Knowledge 


Comprehend 


Application 


Analysis 


Synthesis 


Evaluation 


111 


/ / 


77 








7 


112 


7 








7 




121 


✓ 


7 




77 


7 




122 


7 


7^7 




7 




7 


211 


7 


7 




77 




7 


212 


7 


7 




7 


7 


7 


221 


7 


7 




7 


7 


7 


222 


7 


7 




V 


7 


7 



Many faculty members have found it difficult to apply 
this six-level taxonomy, and some educators have 
simplifiedand collapsed the taxonomy into three general 
levels (Crooks, 1988): The first category knowledge 
(recall or recognition of specific information). The 
second category combines comprehension and 
application. The third category is described as “problem 
solving, transferring existing knowledge and skills to 
new situations. 

Another objective of this study is to determine which 
question categories are easier to implement in an online 
case discussion. The implications may vary if you are 
answering a simple Knowledge category question or 
answering a difficult synthesis question. Therefore, in 
this study we use questions from every category from 
Bloom’s taxonomy. We came up with a list of 40 
different questions. Each one of the eight case versions 
has five totally different questions from multiple 
categories. For example, case version 212 includes one 



knowledge question, one comprehension question, one 
analysis question, one synthesis question, and one 
evaluation question. 

IMPLEMENTATION OF THE STUDY 

This case study was administered to 90 students in a 
juniorcomputer information systems class during Spring 
2001. The student formed their own teams of three 
students each based on their own preferences. Each team 
was assigned a different version ofthe Tradenet case at 
random. Every team had a separate bulletin board area 
for discussion. Students could only see postings from 
their own team members. 

The content ofthe course included 9 short cases. These 
cases were discussed several weeks prior to the 
TRADENET case, therefore, students were already 
familiar with online case study discussion and with 
postingtechniques. Severalversions of the TRADENET 
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case study were posted along with guided questions. 
Data sources for this study include responses to case 
studies, interactions between students, and feedback 
survey results. Data sources consist of transcripts of all 
online case discussions, results of a feedback attitude 
surveys and a computer background survey. 

STUDENT EVALUATION 
OF THE CASE DISCUSSION 

A previously used method to evaluate the effectiveness 
of the case study discussion is to use an evaluation 
questionnaire (Raju et al. 2000, Raju et.al. 1999). A 
student survey was conducted at the end of the semester 
to assess the effectiveness of the online case discussion 
and to evaluate the students’ perceptions of the case 
discussion Student were encouraged to be honest and 
completion of the questionnaire was completely 
optional. A total of74 students completed the evaluation 
questionnaire. Students were also surveyed regarding 
their computer background and Internet skills, as well as 
their attitudes toward online learning. 

TABLE 5 

MEAN RATINGS FOR ITEMS 



ITEM MEAN 

I improved my understanding of basic 

EDI concepts after reading the case. 4.10 

I learned to value other students’ point of view. 3.96 



The first part of the questionnaire asked the respondents 
to indicate the extent of their agreement with 8 
evaluatory statements on a 5-point Likert scale. The 
response scale ranged from 1 which represented the least 
favorable response of strongly disagree, to 5 which 
represented the most favorable response of strongly 
agree. The following table illustrates the overall positive 
reaction to the various aspects of the TRADENET case 
study. The reaction of the students to the case was 
favorable, all questions yielded means of 3.7 or above. 
In other words, the students positively rated all items. 

TABLE 6 

MEAN RATINGS FOR 
KEY DESCRIPTIONS OF CASE STUDY 



THE CASE WAS... 


MEAN 


interesting 


4.17 


relevant 


4.14 


helpful 


4.04 


useful 


4.03 


meaningful 


4.01 


appropriate 


3.99 


valuable 


3.94 


clear 


3.94 


important 


3.93 


practical 


3.90 


engaging 


3.90 


challenging 


3.82 


easy to understand 


3.82 


exciting 


3.49 


difficult 


3.01 



I learned to inter-relate important topics and ideas 



after reading the case. 3.94 

The case improved my ability to integrate. 

technical and managerial issues 3.90 

The case has helped me become more 

confident in expressing ideas. 3.89 

My ability to evaluate critically technical and 

managerial alternatives has improved. 3.83 

I feel my ability to integrate technical and 

managerial issues has improved. 3.81 



ANALYSIS OF DATA 

Every posting was analyzed to determine the quality of 
the contribution and whether participation was sub- 
stantive (Davidson-S hivers et al. 2000). A full analysis 
of each question was made. The criteria used for 
assigning points (grades) to the answers was based on 
the following three levels. 

Level 3: The answer is specific and thorough 

Level 2: The answer has been attempted but the 
answer is clearly lacking in thoroughness 



The case motivated me to learn more about EDI. 3.74 



Level 1: The answer has been attempted 
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The complete statistical of the data will be provided in 
the final version of this paper. 

SUBSEQUENT ANALYSIS 

In a study by LaMaster and Morley (1999) participants 
were encouraged to respond to the cases and questions 
by posting replies. Students enjoyed the general forum, 
with all messagesposted to one main site, wh ile mentors 
preferred small group forums. 

A study by Steinkuehler et al. (2000) compared the 
effects of three forms of online instruction on memory, 
belief change, and argumentation skill. Reading of a 
pro/con text was followed by: (1) online discussion in 
pairs compared to reading of the same text followed by 
two forms of individualized study techniques derived 
from the cognitive memory literature; (2) self- 
explanation; and (3) repeated summarization and study. 
Results were analyzed in terms of argument change from 
pretest to posttest, trans fer of argum ent skills, text rec all, 
reported and directly assessed opinion change, and 
perceptionsofproductivity and participation. Qualitative 
analysis of the transcripts from the online activities 
examined time on task, effects of pair agreement or 
disagreement, andunequal participation within the pairs. 

Other methods used for data analysis include the 
constant comparative method (Murphy et al. 1998). 
Another study by McDonald (1998) focused on the 
following variables. 

the level (quantity) of participation 

the relationship among and between messages 

the functions of the interactions (cognitive, meta- 
cognitive, social, organizational) and determ ine if the 
pattern of those functions changed 

the characteristics and patterns of interpersonal 
interactions 

We will apply content analysis techniques to the 
transcribed discussions in order to examine several 
variables, including level of participation, amount of 
interaction, and cognitive skills. Transcripts of the case 
answers will be analyzed within and across groups using 
a coding scheme based on a content analysis model for 
dimensions of the learning process by Henry (1992). We 
will also generate a setof code words found in each case 
discussion and visual maps (latice graphs) with 



conceptual hierarchies to relate this data. Content 
analysis of computer-mediated communication has been 
used in previous studies of online discussions (Haraet.al. 
2000 ). 

CONCLUSION 

This paper has described the unique aspects of online 
case studies and major advantages and disadvantages. 
The framework for this study has been described in 
detail. Analysis of the data is currently in progress. 
Results from this research-in-progress should provide 
insights about the level of discussion participation in an 
online case analysis, the quality of the discussions in 
online cases. Other issues include interaction and 
learning in an online case study discussion and the role 
of the instructor in the online case discussion 
environm ent. 
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REQUIRING STUDENTS TO BRING 
COMPUTERS TO CAMPUS: ARE UNIVERSITIES 
ACHIEVING THEIR GOALS? 



Cindy Randall 
Georgia Southern University 



ABSTRACT 

According to data from the Campus Computing Project, an independent annual survey of colleges ' information- 
technology plans, less than 10 percent of colleges and universities currently require students to have their own 
computers. (“Growing Number of Colleges 2000) However, as universities fight to secure a competitive 
advantage for both themselves and their graduates in the 21 st century, more institutions will be joining the ranks of 
those currently placing a desktop or a laptop in the hands of each co-ed. The goal of this paper is to poll the current 
thoughts of universities that do have such a requirement in place, of universities that are considering adopting a 
policy requiring computers in the near future, and of universities that have chosen not to jump on this band wagon. 

This paper will explore the technology objectives of these universities and attempt to determine if these objectives 
are being met through universities' current computer ownership requirements. 



INTRODUCTION 

Asthe computer screen replaces the notebook, a growing 
number of universities, both private and public, are 
requiring that students either purch ase or lease their own 
computers, adding at least$ 1 000 per year to the average 
cost of college expenses (Port, 1999). For freshman at 
the University of North Carolina at Chapel Hill, the 
choice was whether or not to pay $21 82 for a standard 
ThinkPad or $2924 for a model with expanded memory 
and a larger hard drive. On some campuses of the 
Polytechnic University in New York, students are paying 
$500 each semester to lease a laptop(“Growing Number 
of Colleges...,” 2000). 

Some schools are requiring the purchase of specific 
hardware and software. Others are supplying minimum 
computer configurations appropriate to different degree 
programs. (University of Florida: Student Computer 
Requirement’s web site, 2001). 

While the students may bear some of the financial 
burden, states are investing heavily to create a 
technology-enabledenvironment. In Massachusetts the 



Board of Higher Education approved a $123 million 
program that includes vouchers forlow-income students 
and training, facilities, equipment, and academic 
programs (“Massachusetts Takes Step..., 2000). The 
plan unconditionally embraces technology at a time 
when educational specialists are still trying to ascertain 
how computer ownership impacts education. The Board 
of Higher Education deemed this to be a necessary step 
in ensuring graduates will have the technological skills 
to fill tens of thousands of business and high-tech job 
openings (Healy, 2000). However, this program has 
stalled for lack of funding and public support. The bo ard 
had anticipated financing the program with state-issued 
bonds. The state IT division ruled against bond 
financing. Because computers rapidlybecomeobsolete, 
monies will have to come out of general revenues. 
Events such as the evaporation of the state’s surplus and 
the public’s reaction to the cost have stalled the Board of 
Higher Education’s plans (Olsen, 200 1). 

Benefits 

All students will have the same academic advantage. 
Typically students who qualify for financial aid are less 
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likely to have the means to purchase computers. If 
technology is a university requirement, the cost can be 
factored into financial aid packages. Some universities 
are using special endowments or have redirected some 
revenue monies to help students meet these costs 
(“Growing Number of Colleges...,” 20 00). 

The network and communications that these computers 
provide can improve faculty/student communications. 
Skills acquired while using the computers would provide 
real skills which would make graduates more marketable 
(Graf, 2000). Software could be obtained at reduced 
prices with a campus site license (Gates, 1998). 

Courses offered in all majors will be able to incorporate 
the use of technology (University of Denver: 2001 
Laptop Specifications web site, 2001). On some 
campuses, freshman are required to attend training 
sessions so thatthey will beable to use and configure the 
computers and will be prepared for class work and/or 
assignments (Lawrence, 1999). 

In the case of wireless LAN technology, students can 
access a course material or e-mail faculty while sitting 
on a bench outside the library. Many believe that the 
greatest benefit of requiring computers comes from 
outside the classroom, where students have 24-7 access 
to campus network and the Internet (Olsen, 2001) 
Technology has elbowed its way out of computer labs. 
Universitiescan reclaim scarcephysical space that once 
housed these labs. No more spending of endless dollars 
to rewire old building with asbestos ceilings and cinder 
block walls to meet changing technology needs. At 
Wake Forest this year, freshman received a wireless 
Ethernet card with their PC notebook, and a bill with 
their tuition statement. Approximately 11 percent of 
Carnegie Mellon’s students purchased wireless LAN 
cards with which to access the university’s wireless 
network (Brewin and Cope, 2000). 

Faced with stagnant enrollment and low SAT scores, 
Western Carolina University implem ented a plan to wire 
campus and require computers in 1995. Students may 
choose to purchase a desktop or a laptop. While 
enrollment has not climbed, the quality of the student 
has. SAT scores are on the rise as WCU emphasizes 
communications skills and critical thinking, not 
computer hardware (Johnson, 2000). 

While most administrators agree that they are unable to 
measure the real impact of computer ownership on 



education, this policy will impact the institution’s 
bottom line. Spending on information technology can be 
greatly reduced for the institution as these costs are 
passed on to the student (Olsen, 2001). 

Concerns 

Not everyone agrees that student ownership of 
computers is necessary. Everyone does agree that 
networks must be improved to accommodate increased 
traffic, the student’s cost of education will rise, and both 
faculty and students will need additional training (Graf, 
2000 ). 

Security and battery life require thoughtand planningfor 
universities that wish to embrace wireless technology 
(Brewin and Cope, 2000). Some faculty also worry 
about the integrity of using high-tech electronic devices 
in the classroom (Kobin, 2000). 

Others are concerned about the added financial burden 
to students (Port, 1999). ChrisDuckenfield, vice provost 
for computing and information technology at Clemson 
University, believes that it is unfair to require students to 
spend money on technology that will be obsolete in two 
years. In 33 labs across campus, Clemson has 800 
computers available that can be configured to suit the 
user. Students do not have to invest in their own 
machine, they do not have to lug machines around, and 
equipment cannot be stolen (“Clemson Develops 
System,” 199 8). 

Students at some universities that require computer 
ownership have been disappointed. The computer 
requirement brought with it the expectation that 
computers would be used in some, if not all, classes. In 
fact, few classes are computer intensive. Restructuring 
courses takes commitment and tim e for faculty mem bers 
to create course content that is electronic (Olsen, 2001). 

RESEARCH QUESTIONS 

One goal of this paper was to survey universities with 
computer requirements and determine what their 
objectives were when these universities decided to 
embrace a technology requirement. The survey 
instrument then asked if these objectives are being met, 
and if not, why not. The instrument also asked the 
universities if they were experiencing any unexpected 
outcomes, good or bad, and if so what these outcomes 
are (i.e., change in enrollment or retention, etc.). 



142 



Proceedings of the 16 h Annual Conference of the International Academy for Information Management 



Another goal of this paper was to poll universities who 
have not begun requiring students to arrive on campus 
with computers in tow. Are they consideringimposing a 
technology requirement? If so, what are the driving 
forces behind this decision? If not, why not? 

A driver behind this project was the strategic planning 
process currently underway at the authors’ university. 
The information technology plan task force has directed 
the university to consider requiring students to 
purchase/lease laptop computers. This requirement 
would supplement or replace the current environment 
which makes 1400 computers at various locations on 
campus available for student use almost continuously 
during academic sessions. In addition, most student 
residence halls are wired for connection to the campus 
LAN. Plans are underway to make older residence halls 
“connected” via wireless technology. 

METHODOLOGY 

Although the authors are primarily interested in the 
computer ownership requirements at peer institutions, 
we decided to use the Chronicle of Higher Education 
listing of colleges/universities as our frame (“How the 
Classification...”, 2000). A stratified random sample of 
20 percent of the frame was selected from doctoral- 
granting institutions (total of 151 extensive, 110 
intensive), master’s colleges and universities (total of 
496 at level 1, 1 1 5 at level II), and baccalaureate colleges 
(total of 228 liberal arts, 321 general, and 57 
baccalaureate/associate). The provost/academic vice 
president at each selected institution was contacted via 
e-mail with a link to a web-based survey. If electronic 
contact could not be made, a letter was sent with an 
enclosed hard copy version of the survey. A follow-up 
contact was made via e-mail/mail approximately two 
weeks after the initial contact. 

RESULTS 

Forty-one private institutions and 59 public institutions 
responded to the survey. These 1 00 responses out ofthe 
295 surveyed renders a response rate of 33 .9 percent. Of 
these 100 colleges and universities, 23 are doctoral- 
granting institutions, 39 are master’s colleges and 
universities, and 38 are baccalaureate colleges. Most (62 
percent) were institutions of 5000 students or less. Table 
1 shows the breakdown of respondents by size of student 
body. 



TABLE 1 



Size of Student Body 


Frequency 


less than 5000 


62 


5000 - 10,000 


19 


10,001 - 19,999 


9 


20,000 - 30,000 


5 


over 30,000 


5 



Thirteen of the 100 institutions that completed the 
survey instrument required students to own or lease their 
own computers. Of those requiring all students to have 
their own computers, 64 percent make this requirement 
for students in every degree program. The remainder of 
those surveyed have this requirement in place for only 
some of their program s. 

One-third allow students to lease computer equipment, 
40 percent require that students purchase computer 
equipment. The remainder of institutions which require 
students to have computers leave the decision to buy 
versus lease up to the students. Two-thirds of these 
schools specify that the computer must be a laptop 
computer. 

Roughly 62 percent of those mandating students must 
have their own computers have had this requirement in 
place for 2 to 3 years. Eighty- three percent believed this 
endeavor has been successful. None of the institutions 
considered the mandate to not be successful. 

Of those not currently requiring computer ownership by 
students, most were not considering changing this policy 
in the near future. For a breakdown of responses, see 
Table 2. 

Chi-square goodness offit tests were conducted for those 
institutions which have computer requirements in place 
to allow a comparison by institution (private versus 
public), type (doctorate-granting institutions, master’s 
colleges and universities, baccalaureate colleges), and 
size of student body. Only one of these tests yielded a 
differencethat can be considered to be significant. Fifty- 
six percent of those schools with a student body 
between!0,001 and 19,999 require students to come to 
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campus with computers. All other categories of schools 
based on size of studentbody had fewer than 12 percent 
of responding institutions mandating that students must 
own or lease a computer (p - .002). 



Table 2 



Future Plans 


Percent 


Currently planning to 
implement computer 
requirement 


3.6 % 


Currently considering 
to implement computer 
requirement 


27.4 


Not considering such a 
requirement 


69.0 



CONCLUSIONS 

This study did not culminate in any definitive results. It 
did provide us with some initial information concerning 
the percentage of institutions that are mand ating stude nts 
arrive with computers and some insight into the 
successes, failures, concerns, and limitations of these 
programs. 

In our survey instrument we asked for a contact person 
at the university who would be willing to talk with us 
and provide a clearerpicture of whatis happeningat the 
institution. Our next step in this research will be to 
contact these people and discuss their decision whether 
or not to mandate that students have computers and the 
reason(s) their institution made this decision. For those 
who chose to require computers, we would like to know 
the goal of their program and whether or not it is being 
achieved. 
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ABSTRACT 

This paper reports the findings of an experiment designed to test extensions of the Technology Acceptance Model 
(TAM) within the context of using the World Wide Web to gather and analyze financial information. The proposed 
extensions are threefold . Based on prior research , cognitive absorption variables are posited as predeterminants 
of ease of use. Similarly, cognitive instrumental variables are used as antecedents of usefulness. Finally a newly 
proposed construct, strategic importance , is used to capture perceptions of application relevance that are not 
personally motivated. While mixed support was found for the cognitive absorption variables , the cognitive 
instrumental variables were found to be strong predeterminants of usefulness . The relationship between usefulness 
and strategic importance was found to be highly s ignificant; however, additional analysis indicated that strategic 
importance was a precursor rather than an effect of usefulness. The effects of adding strategic importance to the 
cognitive instrumental variables is also discussed. 



INTRODUCTION 

The information systems (IS) survey course, as a 
traditional staple of the undergraduate business school 
curriculum, is often the only exposure that non-IS majors 
have to concepts and issues relating to the use and 
management of information technology as a resource. 
Course objectives typically include: 

• Gaining an appreciation for the strategic importance of 
information systems within the business environment; 

• Developing an understanding of information 
technology that enables the student to leverage 
capabilities developed within their own discipline 
when working with information technology 
professionals; and 

• To the extent that a hands-on lab component is 
offered, increasing competencies in office productivity 
tools. 



In theory, these three objectives are interrelated. An 
appreciation for the strategic implications of information 
technology serves as an antecedent motivator for 
students to tackle the more technical aspects of the 
course. Likewise, the hands-on component is designed 
to reinforce technical understanding. 

Two factors, however, may impact the first objective in 
ways that impair the ability to motivate students. First, 
technological advances in business communications, 
most notably those driving enterprise systems, inter-firm 
networks and e-businesses, increase the complexity of IS 
architectures in ways that are difficult to replicate in an 
education setting. Second, non-IS majors (and often, IS 
majors at the beginning of their discipline) are lacking 
not only technical skills, but also business knowledge 
that enables them to frame IS as an important resource. 

Yet an appreciation for the strategic role of IS is more 
necessary than ever, as businesses are using 
communications technologies to open up the “blackbox” 
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of the firm both to customers and suppliers. Non-IS 
business majors are impacted in the sense that they are 
likely to be content providers for internal and external 
web information systems. The ever- widening audience 
for business information increases the responsibility of 
business students as future information providers to 
understand the sources of data, the application software 
used to manipulate and infuse data with meaning, and 
the communications technologies that create access to 
that data. 

The purpose of this paper is to propose a hands-on 
assignment intended to enhance student awareness of the 
Internet as a strategic resource. At present, the 
assignment has been developed as a pilot initiative; 
however, a set of measures based on a modified version 
of the Technology Acceptance Model (TAM) has been 
used to gather preliminary data on its effectiveness in 
changing student intentionality regarding future Internet 
use. The following sections discuss the adoption and 
modification of the TAM model, the measures and the 
implementation methodology within a single classroom 
setting. 

THEORY AND MODEL DEVELOPMENT 

Since its inception in 1989 (Davis), the Technology 
Acceptance Model has proved to be a robust descriptor 
of two factors — perceived usefulness and perceived ease 
of use — leading to user acceptance of new technologies. 
Two aspects of web information systems; content 
(i usefulness ) of the material, and hyper mediation (in 
which ease of use is an important factor) echo the pre- 
determinants of technology acceptance as posited by the 
Technology Acceptance Model (TAM). While this 
study uses an extension of the TAM model as a 
theoretical basis for assessing the value of these 
assignments, it should be emphasized that in this case, 
the focus is not on pure technology acceptance. 
Typically, students have already been using the Internet 
for some time, although largely for entertainment and 
peer-to-peer communications. Therefore, the shift in use 
represents an adaptation of an existing, well-known 
technology to change student perceptions regarding its 
strategic usefulness. Second, the dependent variable of 
user intention, has been separated into two variables to 
capture (1) intent to use web-based technologies for 
gathering and analyzing financial information, and (2) 
increased understanding of web information content as 
a strategic resource. 



Subsequent research (Venkatesh, 1999; Venkatesh and 
Davis, 1996; 1998; Agarwal, Karahanna, 2000) extended 
TAM to incorporate additional pre-determinants of both 
usefulness and ease of use, and to examine the effects of 
intrinsic and extrinsic motivation. For example, 
Venkatesh and Davis (1996, 1999) explored the role of 
computer self-efficacy as an anchor for initial ease of use 
perceptions, finding that an individual’s “comfort level” 
with computers in general affects their perceived ease of 
use before and after direct experience with new 
technology. This suggests that exercises designed to 
enhance general computer self-efficacy are an important 
complement that solely demonstrates systems usability. 

The dramatic increase in web- based applications has led 
to corollary research on the intrinsic motivation 
properties of hyper mediated learning environments. 
Hyper mediated technology— such as that found in 
Internet applications and gaming software — provides a 
holistic experience in which informational aspects are 
augmented by visual cues that create enjoyment and 
“flow.” Again, using TAM as the basis for examining 
possible antecedents of ease of use and usefulness, 
Venkatesh (1999) found that a game-based training 
intervention promoted a higher user acceptance of group 
collaboration software than traditional training. Not 
only was playfulness not a detractor, but also contrary to 
prior TAM research, the effect of perceived ease of use 
on the behavioral intent was greater than the inherent 
usefulness of the collaboration software. 

Agarwal et al., (2000) extended this research by 
examining the effect of cognitive absorption, the state of 
focused and intense involvement with software, as a 
potential mediator of playfulness and personal 
innovativeness as predictors of user acceptance. Using 
survey results of perceptions of the World Wide Web as 
the target technology, cognitive absorption was 
measured as a set of variables, including curiosity , 
control, temporal dissociation, focused immersion and 
heightened enjoyment. The results indicate that play- 
fulness and personal innovativeness were, in fact, fully 
mediated by the cognitive absorption response elicited 
from using the Web. In addition, the inclusion of 
cognitive absorption as direct predictor of behavioral 
intent in a separate test proved significant. 
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With regard to perceived usefulness, Venkatesh and 
Davis (199 8) have posited two separate constructs as 
possible antecedent factors to further delineate the 
perception that a given technology is useful. The two 
constructs reflecting different aspects of a work 
experience that might lead an individual to regard a new 
technology as potentially useful are: 

• Social influences {subjective norms, voluntariness, and 
image) relate to possible effects of adopting a new 
technology on personal interactions within a work 
environment. 

• Cognitive instrumental processes (job relevance, 
output quality, and result demonstrab ility) potentially 
influence an individual’s assessment of the 
effectiveness of a technology in meeting their specific 
job information requirements. 

Using a longitudinal experimental technique, all six 
factors were found to be significant determinants of 
perceived usefulness, although the social interaction 
influences tended to be mediated by direct experience. 

Given that these studies infuse TAM with a richer 
theoretical basis for understanding factors that might 
affect user adoption of a new technology, this study 
assesses a web-based training exercise, incorporating 
aspects of the previous research. Specifically, the 
components relating the cognitive instrumental process 
variables have been selected as having the greatest 
relevance to the objectives of creating a learning 



experience that students will find personally relevant, in 
terms of giving them strategic personal financial 
information. Because the exercise uses a graphical and 
hypermediated environment, the variables measuring 
cognitive absorption are also included. The experiment 
tests the following: 

H, Heightened usefulness and ease of use perceptions 
leads to increases in the intent to use the Web as a 
tool for gathering and analyzing personal financial 
information. 

H 2 Tasks that incorporate hyper mediation heighten the 
properties of cognitive absorption and strengthen 
ease of use perceptions. 

H 3a The focus of using the Web as a tool for gathering 
and analyzing financial information increases user 
perceptions of the dimensions captured by the 
cognitive instrumental process variables, with a 
corresponding increase in perceived usefulness. 

H 3b Heightened usefulness perception of the Web as a 
tool for gathering and analyzing financial 
information also increases the perception of its value 
as a strategic business resource. 

The following model depicts the original Technology 
Acceptance Model relationships and the proposed 
extensions for the cognitive absorption, and instrumental 
variables. 
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METHODOLOGY 

The goal of the exercise was to both introduce students 
to various types of web-based tools that can be used to 
gather and analyze financial information and to convey 
the idea that such tools can provide individuals and 
businesses with a strategic advantage. The three tasks 
were: 

(1) Use of an intelligent search agent to find and 
evaluate various online booksellers. Because the 
search agent canvasses booksellers in Europe and 
Canada as well as the U.S, an ancillary step of the 
task was to verify the exchange rate. 

(2) Use of two decision support tools in making a use 
car purchase decision. The two tools were quite 
different; one is an intuitive graphical interface, the 
second a more familiar text entry style tool. 
Students were instructed on the use of the two tools, 
comparing ease of use and results. 

(3) Acquisition of the student’s own credit report. For 
privacy reasons, this taskwas entirely voluntary, and 
the assessment of the students is limited solely to 
their perception of amount and extent of information 
available. 

The empirical test of the exercise involved a single 
computer lab session conducted across multiple 
undergraduate student groups. Half of the students 
received the treatment exercise, and the other half a 
trivial web information gathering exercise. All students 
were given a survey questionnaire immediately upon 
completing the instructional session to determine initial 
levels of computer self-efficacy, and to establish baseline 
results on the parameters of cognitive absorption and 
cognitive instrumental variables. 

The survey questions to determine levels of student 
cognitive absorption were adapted from a similar 
instrument used by Agarwal and Karahanna (2000) and 
include their five variables. The remaining questions 
elicited student perceptions of the cognitive instrumental 
variables; perceived usefulness, ease of use, and 
intentionality as derived from Venkatesh and Davis 
(1998). Finally, three items measuring perception of the 
strategic importance of the web as a financial analysis 
tool were developed specifically for this exercise. All 
items were measured on a seven point scale with 
strongly agree and strongly disagree as the endpoints. 



An initial analysis of the instrument psychometric 
properties indicated acceptable levels of internal validity 
for the scales measuring the cognitive instrumental 
variables, perceived ease of use, usefulness, intention 
and strategic importance. The scales developed for the 
three of the cognitive absorption variables (temporal 
disassociation, heightened enjoyment and control) 
demonstrated at least one item with low internal 
correlation. In examining these items further, it was 
found that the two items for temporal disassociation 
(“Most times when I get on the Web, I end up spending 
more time than I had planned", ‘7 spent more time on 
the Web assignment than I had intended ") measured a 
slightly different dimension — that of devoting more time 
to the activity than originally intended. While this 
dimension was appropriate in the Agarwal study which 
examined general web use, the items did not fit the 
experimental setting with its specified assignment. Each 
of the problematic items for heightened enjoyment and 
control were the result of reverse coding. Dropping 
these items substantially improved Chronbach alpha 
coefficients (Table 1). The resulting scales are included 
in Appendix A. 

TABLE 1 

SCALE RELIABILITIES 



Scale 


Number of 
Items 


Reliability 


Behavioral Intention 


3 


.758 


Perceived Usefulness 


4 


.869 


Perceived Ease of Use 


4 


.799 


Strategic Importance 


3 


.876 


Information Quality 


2 


.769 


Result Demonstrability 


4 


.806 


Personal Relevance 


3 


.782 


Temporal Dissociation 


3 


.819 


Focused Immersion 


5 


.772 


Heightened Enjoyment 


3 


.924 


Control 


2 


.764 


Curiosity 


3 


.837 



DATA ANALYSIS AND RESULTS 



To gauge the effect of each variable individually, single 
variable regressions were conducted for each of the 
cognitive absorption and instrumental variables on the 
corresponding TAM variable and on the independent 
variable of strategic importance. Multiple regressions 
were also conducted to ascertain the composite effect of 
the independent variables. 
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Hypothesis H,, which posits ease of use and usefulness 
as antecedents of intention to use a given technology, is 
a straightforward application of the TAM in the context 
of using the web to gather and analyze financial 
information. The results indicated mixed support. 
Individually, both usefulness and ease of use were 
shown to be significant predictors of the intention to use 
the web (Table 2). The composite model was also 
highly significant (p < 0.0001), with an R-squared of 
0.584; however, ease of use switched a positive to a 
negative affect on behavioral intention. 

With regard to the cognitive absorption variables (H 2 ), 
the results of the single regressions (Table 3) indicate 
that three of the variables: temporal dissociation, 



heightened enjoyment, and control were positive 
determinants of strengthened perceptions of ease of use 
within the context of gathering and analyzing financial 
information. In a multiple regression assessing the 
composite model, however, only temporal disassociation 
and control remained significant. Curiosity became 
highly significant, but in the opposite direction of the 
hypothesized relationship. While the composite model 
was significant, it explained only 30% of the variation in 
perceived ease of use. These findings only partially 
support hypothesis H 2 that tasks incorporating 
hypermediation heighten the properties of cognitive 
absorption with a commensurate effect on ease of use 
perceptions. 



TABLE 2 

RELATIONSHIP OF PERCEIVED EASE OF USE, USEFULNESS WITH INTENTION TO USE 





Individual Variable Effects 


Composite Model Effects 




p 


t 


p 


t 


Perceived Ease of 
Use 


.527 


5.5 7*** 


-.228 


-1.94* 


Perceived 

Usefulness 


.758 


10.65*** 


.935 


-8.15*** 


Effect of Ease of 
Use on Usefulness 


.807 


12.07*** 




R 2 = 0.5 84*** 


*p < 0.10 


**p <0.05 


***p < 0.01 






TABLE 3 

RELATIONSHIP OF COGNITIVE ABSORPTION VARIABLES W ITH PERCEIVED EASE OF USE 




Individual Variable Effects 


Composite Model Effects 




p 


t 


p 


t 


Temporal Dissociation 


.237 


3.43*** 


,.175 


1.66* 


Focused Immersion 


.088 


1.00 


-.077 


-0.86 


Heightened Enjoyment 


.204 


3.16*** 


.073 


0.57 


Control 


.377 


4.98*** 


.428 


3.56*** 


Curiosity 


.102 


1.35 


-.319 


-2.77*** 










R 2 = 0.3 02*** 


*p < 0.10 


**p <0.05 


***p < 0.01 






Proceedings of the l6 h Annua 


l Conference of the Interne 


ttional Academy for 


Information Management 


73 




89 



Hypothesis H 3a , that the cognitive instrumental variables 
of result demonstrability, information quality and 
personal relevance are positive determinants of 
usefulness was fully supported (Table 4). It should be 
noted that in further analysis of the composite model, 
personal relevance was both the most highly significant 
and largest predictor of change in the perception of 
usefulness. 

However, the fourth hypothesis (H 3b ), that of usefulness 
as a pre-determinant of perceptions of strategic 
importance presented the most interesting findings. In 
multiple regression modeling usefulness, ease of use and 
strategic importance, the relationship between usefulness 
and strategic importance was found to be highly 
significant (Table 5). However, further analysis of the 
relationship between usefulness, strategic importance 
and behavioral intention indicated a fully mediating 
effect of usefulness on strategic importance (Table 6). 



Of equal interest (Table 7) was a corollary analysis of 
strategic importance added to the cognitive instrumental 
variables as determinants of usefulness. This model, 
with an R-squared of 0.767 was highly significant (p < 
0.0001). Although all four determinants remained 

significant, result demonstrability (p < 0.087) and 
information quality (p < 0.067) were no longer 

significant at the alpha = 0.05 level. Strategic 

importance and personal relevance both continued as 
highly significant (p < 0.0001), indicating partial 
mediation of the effects of result demonstrability and 
information quality. Similarly, the beta coefficients for 
strategic importance and personal relevance 
demonstrated that they were stronger predictors of 
variation in usefulness. 

Finally, although not formally tested as a hypothesis, 
two experimental treatments were given: one with web 
exercises directed towards gathering and analyzing 



TABLE 4 

RELATIONSHIP OF COGNITIVE INSTRUMENTAL VARIABLES WITH USEFULNESS 





Individual Variable Effects 


Composite Model Effects 




p 


t 


p t 


Result Demonstrability 


.747 , 


9.99*** 


.214 2.29** 


Information Quality 


.641 


8.41*** 


.214 2.91*** 


Personal Relevance 


.810 


12.78*** 


.513 5.56*** 








R 2 = 0.7 16*** 


*p < 0.10 


**p <0.05 


***p < 0.01 




TABLE 5 

RELATIONSHIP OF EASE OF USE AND USEFULNESS WITH STRATEGIC IMPORTANCE 








Composite Model Effects 






p 


t 


Usefulness 




.634 


5.76*** 


Ease ofUse 




-.003 


-0.03 








R 2 = 0.506*** 


*p < 0.10 


**p < 0.05 


***p < 0.01 
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TABLE 6 

RELATIONSHIP OF EASE OF USE AND USEFULNESS WITH STRATEGIC IMPORTANCE 





P 


Composite Model Effects 

t 


Usefulness 


.643 


6.41*** 


Strategic Importance 


.182 


1.61 






R 2 = 0.579*** 


*p<0.10 **p <0.05 


***p<0.01 \ 

TABLE 7 





RELATIONSHIP OF COGNITIVE INSTRUMENTAL VARIABLES AND 
STRATEGIC IMPORTANCE WITH USEFULNESS 



Individual Variable Effects Composite Model Effects 





p 


t 


p 


t 


Strategic Importance 


.801 


9.43*** 


.333 


4.27*** 


Result Demonstrability 


.747 


9.99*** 


.150 


1.73* 


Information Quality 


.641 


8.41*** 


.130 


1.85* 


Personal Relevance 


.810 


12.78*** 


.445 


2.77*** 



R 2 = 0.7 67*** 

*p<0.10 **p <0.05 ***p<0.01 



personal financial information; the other derived from 
the Laudon and Laudon text. The latter included 
exercises to check United Postal Service shipping rates 
and to determine hotel conference rates. An ANOVA of 
the two groups did not uncover any significant 
differences for the TAM, cognitive instrumental or 
strategic importance variables, but both heightened 
enjoyment and curiosity did indicate variation between 
the groups. In both cases, the group receiving the 
personal exercises experienced greater enjoyment and a 
higher level of curiosity than the group performing the 
business oriented tasks. 

DISCUSSION 

In retrospect, this study cast a wide net by examining 
predeterminants of both perceived ease of use as 
variables measuring cognitive absorption, and usefulness 
in the form of the cognitive instrumental variables in the 



same experiment. These predeterminants captured very 
different aspects of the application exercise. Again, it is 
important to note that the focus of the exercise was quite 
narrow — that of using web-based tools for gathering and 
analyzing financial information. Within this context, the 
cognitive absorption variables measured whether the 
web based hypermediation created an engaging mental 
environment for accomplishing the task. The useful- 
ness component turned on the quality and relevance of 
the tools. 

With regard to the relationship between cognitive 
absorption and ease of use, there was some support of 
Agarwal’s original findings. Temporal disassociation 
and control were found to be significant determinants, as 
was curiosity, although in the opposing direction. Given 
that Agarwal’s study examined general web use, and the 
experiment conducted in this assessment consisted of a 
controlled assignment, it is not surprising that 



Proceedings ofthe l6 h Annual Conference of the International Academy for Information Management 



75 



heightened enjoyment and focused immersion would not 
play a significant role, as they might if the subjects were 
left to “surf the net” at will. 

Of the three factors that were found to be significant, 
control emerged as having the greatest effect. Again, 
this is consistent with the nature of the exercise, as the 
ability to control (through the graphical interface and 
user-directed hyperlinks) the various web-based tools to 
complete the assignment should convey both a sense of 
mastery and corresponding ease of use. Temporal 
disassociation, while significant only at the p < 0.10 
level, was also consistent with AgarwaTs findings. 

The negative results for curiosity were surprising; 
however, the findings may be explained by the nature of 
the questions posed by the measurement instrument. As 
with heightened enjoyment and focused immersion, the 
constructs captured by: “I was stimulated.. ..excited ...my 
imagination was stimulated ” are more applicable to a 
situation in which the individual is roaming the web at 
will, rather than completing a business task. It should 
also be noted that because group differences were found 
for this variable, the composite model was assessed 
using multiple regression for each group. While both 
generated a negative coefficient, the group receiving the 
personal exercises was not significantly different from 
zero. Therefore, the significance of the negative findings 
was driven solely by the second group that received a 
standard set of business web-exercises based on the 
Laudon and Laudon text. 

The importance of this study is found in the relationship 
between the cognitive instrumental variables, strategic 
importance, and usefulness. As hypothesized, the 
cognitive instrumental variables were found to be 
significant predeterminants of usefulness, replicating the 
findings of Venkatesh. The anticipation was that 
usefulness would be a predeterminant of the perception 
of strategic importance; and the initial findings did 
indicate a significant relationship between the two. 
Similar assessments of strategic importance and the 
intention to use also detected a significant relationship. 
But additional analysis of the model with intention to use 
included as a third variable, provided unexpected results. 
While usefulness remained a significant antecedent of 
intention to use, strategic importance was no longer 
significantly different from zero. Clearly, usefulness 
mediated the relationship between perception of strategic 
importance and intention to use. 



With an improved understanding of the role of strategic 
importance as a predeterminant ofusefulness; analysis of 
the composite model including the strategic importance 
presented interesting results. While each of the 
cognitive instrumental variables remained significant, 
the two strongest factors driving beliefs of system 
usefulness were (1) perceptions of personal relevance, 
and (2) strategic importance. The addition of strategic 
importance also increased the explanatory value of the 
composite model from an R-squared of 0.715 for the 
three instrumental variables in isolation, to an R-squared 
of 0.767 for the model encompassing both the 
instrumental variables and strategic importance. This 
appeared to indicate that there may be a dimension of 
usefulness that is impacted by considerations of 
relevance extending beyond a preoccupation with 
personal productivity. 

CONCLUSION 

Given that the study was largely a replication of prior 
research involving a different application of technology, 
the findings do provide mixed support for cognitive 
absorption as a predeterminant of ease of use, and 
strongly support Venkatesh’s results of cognitive 
instrumental variables as antecedents of usefulness. 
There are, of course, limitations. Students are 
somewhat notorious as unreliable subjects. The 
instrument items, while maintaining internal validity, 
may have not been adequately adapted to the focus of the 
particular exercise. 

Nevertheless, the findings regarding strategic importance 
as an antecedent of usefulness, and as a construct that is 
separate from personal relevance, is an important finding 
with interesting implications for practical applications. 
As businesses move towards distributed information 
technologies, such as enterprise resource planning 
systems, a strategic understanding of a system’s 
usefulness could possibly mediate a reluctance to use a 
system — particularly if it appears that personal 
productivity may suffer. If this is the case, training 
interventions that emphasize the strategic importance of 
a particular technology could be developed to overcome 
personal reluctance, thereby improving user acceptance. 
A rigorous development of strategic importance as a 
component of user beliefs and intention to use, may 
enlarge our understanding of this process, perhaps 
increasing its effectiveness. 
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APPENDIX A 
MEASUREMENT SCALES 



Cognitive Absorption 
Temporal Dissociation 

1 . Time appeared to go by very quickly while I was working on the Web assignment. 

2. I lost track of time while using the Web. 

3. I found that the time flew by during the Web assignment. 

Focused Immersion 

1 . While completing the Web assignment, I was able to block out most other distractions. 

2. While completing the Web assignment, I was absorbed in what I was doing. 

3. I was immersed in the task of completing the Web assignment. 

4. While completing the Web assignment, I was not easily distracted by other attentions. 

5. While on the Web, I found my attention was not easily diverted. 

Heightened Enjoyment 

1. I had fun interacting with the Web for this assignment. 

2. Using the Web for this assignment provided me with a lot of enjoyment. 

3. I enjoyed this Web assignment. 

Control 

1 . When using the Web for this assignment, I felt in control of the material. 

2. Using the Web for this assignment allowed me to control my computer interaction. 
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Curiosity 

1 . Using the W eb for this assignment excited my curiosity. 

2. My interaction with the Web during this assignment made me curious. 

3. Using the Web to complete this assignment stimulated my imagination. 

Information Quality 

1 . The quality of financial information I can gather and analyze on the Web is high. 

2. I have no problem with the quality of financial information I can obtain and analyze on the Web. 

Result Demon str ability 

1. I have no difficulty telling others about the results of using the Web to gather and analyze financial 
information. 

2. I believe I could communicate to others the advantages/disadvantages of using the Web to gather and analyze 
financial information. 

3. The advantages and disadvantages of using the Web to gather and analyze financial information are apparent 
to me. 

4. I would have no difficulty explaining why using the Web to gather and analyze financial information may 
or may not be beneficial. 

Personal Relevance 

1. Using the Web to gather and analyze financial information gives me an important personal advantage. 

2. Using the Web to gather and analyze financial information is relevant to my personal financial decisions. 

3. I believe the information I obtain and analyze through the Web is helpful in making personal financial 
decisions. 

Perceived Ease of Use 

1. I find the Web easy to use as a tool to gather and analyze financial information. 

2. Interacting with the Web to gather and analyze financial information does not require a lot of my mental 
effort. 

3. I find it easy to get the financial information I want using the Web. 

4. My interaction with the Web as a tool to gather financial information is clear and understandable. 

Perceived Usefulness 

1 . Using the Web improves my ability to gather and analyze financial information. 

2. Using the Web to gather and analyze financial information enhances my effectiveness in making decisions. 

3. I find the Web to be useful to me in gathering and analyzing financial information. 

4. Using the Web-based increases my productivity in gathering and analyzing financial information. 

Behavioral Intent 

1. Assuming access to the Web, I intend to use it for gathering and analyzing financial information. 

2. Given access to the Web, I predict that I will use it to gather information for making financial decisions. 

3. I believe that I will continue to use the Web for information when faced with financial decisions. 

Strategic Importance 

1. I understand that using the Web to gather and analyze financial information could give businesses an 
important strategic advantage. 

2. I understand that using the Web to gather and analyze financial information could be relevant to making 
important business decisions. 

3. I understand that the Web can provide important tools that could help businesses gather and analyze 
information to their strategic benefit. 
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ABSTRACT 

Distance Learning (DL) initiatives are proceeding full speed ahead , both within traditional universities and in 
“virtual ” institutions specializing in on-line course delivery . Much has been written about the virtues and limitations, 
the obstacles and enablers, and the “Do's” and “Don’ts” of DL. However, considerable work remains in 
determining actual learning outcomes of the various approaches to DL. This paper describes a useful framework for 
designing and evaluating DL course delivery that is based on Maslow } s Hierarchy of Needs model applied to virtual 
learning communities. The results of analyzing preliminary outcomes from a set of virtual courses in Introduction 
to Information Systems provide some initial support of the framework. 



INTRODUCTION 

A number of factors have contributed to the growth and 
the phenomenon of Distance Learning (DL). University 
administrators are under continued pressure to increase 
enrollments, to decrease costs by serving more students 
with fewer faculty and university resources, and to make 
education more accessible to a wider range of learners, 
both geographically and demographically. Techno- 
logical advances in software tools, data communication 
networks, and multiple media, most dramatically 
demonstrated in the exponential growth of the World 
Wide Web, have provided the infrastructure for making 
it possible to achieve these goals. Continued increases 
in existing and immigrant populations, strong economic 
growth in many regions, and a growing awareness of the 
economic benefits of a college education, have all 
combined to increase demand for post-secondary 
education from a widening diversity of demographic 
sources (Dede 1990). 

The result is that DL initiatives are proceeding full speed 
ahead, both within traditional universities and in 
“virtual” institutions specializing in on-line course 



delivery. Much has been written about the virtues and 
limitations, the obstacles and enablers, and the “Do’s” 
and “Don’ts” of DL. However, considerable work 
remains in determining actual learning outcomes of the 
various approaches to DL. This paper describes some 
relevant background and the subsequent evolution of a 
useful framework for designing and evaluating DL 
course delivery. The results of analyzing preliminary 
outcomes from a set of virtual courses in Introduction to 
Information Systems provide some initial support of the 
framework. 

BACKGROUND 

What is the definition of “Distance Learning?” DL 
means different things to different people. Its definition 
has evolved as computer and communications 
technologies have become more sophisticated, more 
powerful, less costly, and more widespread. For the 
purpose of this paper, DL is defined as course delivery 
over networks to multiple geographic locations using 
flexible access methods. The term DL is used 
synonymously with the term on-line learning, and also 
refers to what is sometimes called asynchronous 
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learning, although DL can be both synchronous as well 
as asynchronous depending upon the tools used. 

Distance Learning and Education Theory 

The current state of DL represents the convergence of 
technical and social educational evolution. Pre-computer 
correspondence courses are often cited as the earliest 
examples of DL (Dunning 1990). However, the most 
traditional type of DL is that of the one-way video 
transmission of a lecture format to one or more classes of 
remotely located respondents. Advances in technology 
have facilitated changes in the perception and definition 
of DL from a passive 1-way video lecture model to an 
interactive student-instructor and student-student model 
employing a variety of communication media as needed 
(Bates 1991). 

Education theories based on Vygotsky (Vygotsky 1978) 
and others have served as the foundation for the concept 
that effective learning must be active, cooperative, 
student-centered, and socially constructed (Damon 
1984); (Topping 1992); (Webb 1982). A body of 
research known as computer-supported cooperative 
learning (CSCL) initially focused on the application of 
information technologies to individual students and 
small groups in a traditional classroom context 
(Koschmann 1996). This research stream offers insight 
into the now extensive use of the WWW and 
accompanying Internet tools to deliver interactive DL. 
At the same time, studies of computer-mediated 
communication (CMC) provide useful information about 
the effective use of various electronic media now used in 
delivering interactive instruction (Berge and Collins 
1995). 

Learning Communities 

As the focus of computer-support for learning has 
evolved from individual interaction with a computer 
toward technology support for interaction among 
dispersed members of the group, an additional 
development in educational research and practice has 
become increasingly relevant: the learning community. 

A learning community encompasses the idea that 
learners benefit more if they actively contribute to their 



own and others’ learning. The members have a socially 
constructed, shared view of their goals and processes, 
and a sense of identity and belonging to the community. 
A virtual community is a real community but whose 
members interact electronically as well as, or, for some 
or all members, in place of physical interaction. A 
virtual learning community incorporates both the 
primary goal of learning and the notion that at least some 
members of the community do not interact face-to-face 
(FTF) and thus require various technologies to 
communicate (Hiltz 1 994). 

Although some DL research has focused on the pros and 
cons of particular CMC tools (e-mail, bulletin boards, 
chat), others have taken a more holistic view, 
demonstrating that a DL class can indeed result in an 
effective learning community (Powers and Mitchell 
1997); (Haythomthwaite, Kazmer et al. 2000). An 
effective DL environment provides a flexible toolset that 
allows the instructor to adapt the appropriate tools to 
various pedagogical methods and to create a learner- 
centered environment by identifying what knowledge 
and skills the student brings to the course and configures 
the environment to meet her there (Bransford, Brown et 
al. 2000). It also allows the students to select tools that 
match the various types and levels of interaction 
(public/private, informal/formal, spontaneous/reflective) 
needed in developing a virtual learning community 
(VLC), as well as the tools that can best exploit the skills 
and knowledge they bring to the class. 

The purpose of this research is to address the following 
question: How can our understanding of virtual learning 
communities help us to design, and to evaluate the 
effectiveness of DL experiences that support 
pedagogical goals and contribute to student learning 
outcomes? 

RESEARCH FRAMEWORK 

Maslow (1970) developed a Hierarchy of Needs, which 
describes five layers of needs that humans experience: 
physiological needs, safety needs, the need for love and 
belonging, esteem needs, and the need for self- 
actualization (Figure 1). Lower level needs must be 
fairly well met before an individual experiences higher 
level needs. 
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FIGURE 1 

MASLOW’S HIERARCHY OF NEEDS 




According to the theory, the first four levels represent 
deficit needs: an individual feels the need if he does not 
have enough. After a need is met, it is no longer felt. 
After one’s needs are more or less met on one level, a 
person feels the needs on the next level. For example, if 
an individual has no food or water and no friends, he 
will first try to get enough food and water before seeking 
friends. Self-actualization, or growth motivation, is not 
a deficit need; rather, it is continually felt. Once 
experienced, the need to fulfill one’s potential to the 
fullest does not stop. All deficit needs must be fairly 



well met before a person can be self-actualizing. Self- 
actualized people are independent, yet have intimate 
personal relationships with a few people. They are 
compassionate and creative, and accept themselves and 
others for what they are. Few people are self-actualizing 
(Maslow 1970). 

This Hierarchy of Needs can be applied to community 
building on the World Wide Web (Kim 2000). Table 1 
summarizes this concept. Members of a virtual 
community go through a similar set of levels of need in 
order to become active, contributing members of the 
virtual community. A participant must first be able to 
gain access to the community, through technology and 
training (knowing how to access it). Once able to take 
this for granted, an individual experiences the need to 
feel safe from unwanted or unexpected negative 
interactions from other community members. The next 
level is where she will successfully interact with other 
community members and develop a sense of identity 
with the community. Once this need is met, she is able 
to move on to making positive, acknowledged 
contributions to the community. The highest level, 
which is likely to be achieved only by a small proportion 
of community members, involves taking on recognized 
community roles and helping to shape the continuing 
evolution of the community, which in turn helps the 
individual develop. 



TABLE 1 

ONLINE HIERARCHY OF NEEDS 1 



Level of Need 


Maslow’s Definition 


Corresponding Online 
Learning Need 


Physiological 


Oxygen, water, food, clothing, 
shelter, health 


System and Internet access 


Safety 


Safe circumstances, security, 
protection, stability, structure, 
order 


Protections from personal 
attacks and hackers. 
Structure, order, consistency 


Belonging 


Ability to give and receive love; 
need to be a member 


Belonging to a community 
(the class as a whole) and to 
subgroups (e.g., teams) 


Esteem 


Status, recognition, attention, self- 
respect, confidence, competence, 
achievement 


The ability to contribute to the 
course/community and be 
respected for it. Competence 
and achievement. 


Self-actualization 


Ability to fulfill one’s potential 


The ability to take a role in the 
course/community that 
develops and challenges one’s 
self. 



1 Adapted from Kim, “Building a Successful Community” (p. 9). 
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How can educators apply this model toward building 
virtual learning communities in distance learning 
classes? Students who are having trouble accessing the 
systems, either due to technical problems or lack of 
training, cannot be expected to move on to higher levels 
of participation until these issues have been resolved. 
Likewise, if students do not feel safe from attacks or 
criticisms when making contributions, they will not be 
able to become contributing members of discussions or 
sub-groups (teams). Finally, students who do not feel a 
sense of belonging to the community or sub-group are 
less likely to contribute, and thus likely to get less from 
the course. 

The on-line hierarchy describes how students can move 
from being individual learners to becoming active 
members of a learning community, through knowledge 
and assessment. Because students entering a course may 
come from a variety of backgrounds and perspectives, 
DL instructors should at least initially emphasize a 
learner-centered approach. Ongoing assessment will 
provide feedback to the instructor and students about 
their need levels in the on-line hierarchy, as well as in 
their knowledge in the course domain. Students who 
have unmet needs at lower levels are likely to perform 
more poorly until those needs are met and they can move 
on to become more active, participating members of the 
learning community. 

RESEARCH METHODS 

In order to gain some initial support for the framework 
described above, data were collected from two distance 
learning sections of an undergraduate Introductory 
Information Systems class. There were 52 students 
enrolled in the two sections, 28 in one section and 24 in 
the other. Text, on-line notes, assignments, projects, 
grading criteria, and exams were identical in both 
sections. Each student completed five individual 
homework assignments, including three “hands-on” 
assignments (constructing a spreadsheet, database, and 
a web store) and two written exercises (on expert 
systems and networks). Each class had its own 
WebBoard where questions were posted and discussed 
by the class as a whole. Participation in these 
discussions was required. 

Students in each class were randomly assigned to a team 
at the start of the course and virtually no one knew their 
teammates before the class. Each team was given a 
private discussion area on the WebBoard, which they 
could use for communication. However, no restrictions 



were placed on communication methods — teams could 
meet face-to-face if they wished. Two team projects 
were required. There was a short hardware configura- 
tion assignment due within the first few weeks of the 
course, designed to help teams get used to working 
together. The second project involved researching a 
topic, and then designing and constructing a web site to 
educate the class about the topic. 

Although WebCT was the primary online tool used in 
both sections, technological limitations prevented all its 
features from being used. Students took online quizzes 
covering each chapter in WebCT and had the option to 
first take practice quizzes. Narrative lecture notes were 
available online for each chapter, explaining material 
that the instructor would have covered in a lecture. Due 
to the technological difficulties, the asynchronous 
conferencing facility, e-mail, and chat features of 
WebCT were not used. Instead, the classes used external 
e-mail, and O’Reilly & Associates, Inc. WebBoard for 
web conferencing (primarily through asynchronous 
postings, although the realtime chat feature was available 
for student teams to use). 

RESULTS 

Several items were included on the course evaluation 
administered at the end of the semester to determine 
student perceptions of the various online tools used in 
the course (Table 2). Each item was answered on a scale 
of “1” (Poor) to “5” (Excellent). Thirteen students 
answered the questions. 

As can be seen in Table 2, students believed that the on- 
line quizzes (both practice quizzes and graded quizzes) 
contributed most to their learning, followed closely by 
individual assignments (i.e. homework). Both 

WebBoard discussions and team projects were 
considered less than “good” in helping students learn. 
The nonparametric Mann-Whitney test was used to 
identify differences in response between the two 
sections; no significant difference was found 
(significance for all tools except WebBoard was greater 
than 0.80; WebBoard was p - 0.29), indicating 

responses were not affected by the class or instructor. 

As shown in Table 2, the more familiar and structured 
tools were preferred. That is, practice quizzes, 
homework, and lecture notes, familiar tools to students, 
were preferred over the less structured and less familiar 
Webboard discussions and collaborating with a team, 
largely by electronic means. 
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TABLE 2 

COURSE EVALUATION RESULTS 



Item 


Mean* 


S.D. 


How helpful were the WebCT On-Line Ouizzes in helping you learn the course 
material? 


3.93 


0.83 


How helpful were the individual assignments in helping you learn the course material? 


3.71 


0.85 


How helpful were the WebCT Notes in helping you leam the course material? 


3.29 


0.99 


How helnful were the WebBoard Discussions/Conferences in helping you leam the 
course material? 


2.94 


1.29 


How helpful were the Team Projects in helping you leam the course material? 


2.88 


1.20 



Students were also asked how their team communicated 
while working on assignments, reporting the proportion 
of communications taking place face-to-face, via phone, 
using e-mail, on the WebBoard, or using online chat. An 
additional analysis was performed which linked team 
project outcomes (grades) to the their usage of these 
CMC tools. Although teams could use any means to 
communicate (in fact, there were scheduled optional 
opportunities to allow them to meet on campus), all but 
one team chose e-mail or the WebBoard for most of their 
communication. In general, teams who relied almost 
completely on e-mail to communicate received lower 
grades than did those who supplemented e-mail with 
asynchronous web conferencing and other media. 

DISCUSSION 

How do these results contribute to our understanding of 
designing virtual courses and learning communities? In 
the context of the hierarchy, it makes sense that some 
students preferred the more structured and familiar tools, 
and were hesitant about WebBoard participation and 
virtual teamwork. This was the first distance learning 
class for almost half of the students. Most of the 
remaining students had taken only one previous distance 
class; a few had taken more. Thus, some of the students 
were still trying to understand the technology and make 
it work; others were seeking order and possibly feared 
personal attacks. 

Since most people have not reached Maslow’s self- 
actualizing level, educators must understand that most 
students are struggling to meet deficit needs 
(physiological, belonging, esteem). Instructors need to 
provide them with tools and pedagogy to help them meet 
these needs. This meshes well with current learning 
theory that learner-centered environments are optimal. 
Learner-centered environments diagnose where the 



student is and start from that point (Bransford, Brown et 
al. 2000). In short, students cannot be expected to 
engage in lively real-time debates when they cannot 
maintain a constant connection to their ISP. Many of the 
students in these two sections were still trying to figure 
out how a distance class works, as well as simply how to 
get on-line consistently. Therefore, it is not unexpected 
that they preferred the most structured and familiar tools: 
quizzes and notes. 

The results from the team project analysis suggest that 
students who relied primarily on e-mail communication 
may have been lower in the hierarchy of needs and this 
resulted in lower performance overall. A tentative 
conclusion might be that those students who performed 
better had moved to higher levels and were able to make 
more effective and appropriate use of the variety of tools 
available. 

CONCLUSIONS AND FUTURE RESEARCH 

Clearly, the data discussed here was collected from post- 
hoc personal reports from a very limited sample and can 
only be used to suggest directions for further research. 
The sample consisted of general business students taking 
a required Information Systems course. However, the 
model offers much potential in guiding our design and 
evaluation of DL courses. DL courses need to determine 
and address the unmet lower level needs of incoming 
students (many institutions now offer self-assessments 
which contribute to this). In addition, students and 
instructors need flexible toolkits that offer them a 
graduated progression from more comfortable to more 
challenging tools. Finally, the virtual learning 
community approach, enriched by the on-line hierarchy 
of needs and with a learner-centered emphasis, provides 
a rich context for design and assessment of DL. 
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Extended Abstract 



Electronic markets are becoming increasingly popular 
and are providing companies with another avenue in 
which to compete for revenue. The rise in popularity of 
the Internet combined with the increasing processing 
capabilities of computers and increasing computer 
literacy of users has added fuel to the e-commerce fire. 
New startups and traditional brick and mortars are using 
e-commerce to increase transaction effectiveness and 
efficiency (Lee and Clark, 1996-1997) and enter new 
markets. Due to the increasing usage of the Internetand 
e-commerce, student demand for e-commerce courses 
has increased dramatically at universities as seen by the 
increase in number of courses offered. Similar to the 
O’Hara and Stephens (1999) research on a required IT 
course, this study “should be a useful bench marking 
tool for those teaching such courses, for those 
responsible for IT curriculums, and for those involved in 
AACSB accreditation or reaccreditation .” In an attempt 
to assist e-commerce course designers in selecting 
content and methodologies, this article will summarize 
the analysis of the e-commerce courses offered at top 
MBA programs across the United States. 

Schneberger, et al. (2000) and Surendra (2000), among 
others, have attempted to outline e-commerce curricula. 
Although these professors have compiled some pertinent 
materials, the basis for their curricula has been largely 
based upon peer discussions and personal opinion. In 



many instances this may be a valuable means of 
designing a course, but it is the authors’ opinion that an 
evaluation of e-commerce courses taught at top MBA 
programs would provide a better method of collecting 
topics, methodologies, and ideas related to a course. 

The sample consists of schools that are listedin Business 
Week’s ranking of top business schools. The top thirty 
schools were chosen and an initial search of the School 
web pages was performed. In cases where the course 
syllabus could not be found, the Dean’s office was 
contacted with a request for the name of the professor 
teachingthe introductory level e-commerce course at the 
MBA level. A fax was sentto that professor requesting 
a copy of the syllabus. Syllabi from eighteen of the thirty 
universities were received. 

i 

General domains of knowledge were chosen from the 
sample. Each course was analyzed with the purpose of 
dividing all content into the domains previously 
determined. After the initial domain classification, a 
second analysis was performed to determine any sub- 
domains in existence. Based upon the preliminary 
research performed, domains include: 

• Business models 

• Strategy 

• Infrastructure 
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• Technology 

• Programming languages 

• Privacy and security 

• Payment 

• Advertising 

• Pricing 

• Auctions 

• Legal issues 

Earlier work by Sendall (1999) found 18 academic 
domains, many of which were similar to those found in 
this study. 

After completing the classificationof all content covered 
in the e-commerce courses, an outline will be developed 
that will provide other programs with the framework 
around which to build an e-commerce course. As 
universities expand their course offerings to include e- 
commerce, they can learn from other universities that 
already have courses in existence (O’Hara, 2000). This 
study will hopefully lead to a standard model that can be 
used and is based upon the content covered in some of 
the best schools in the world. 
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ABSTRACT 

The paper describes a research that investigates the alignment between e-business academic programs and skill sets 
that are sought after in e-business related job postings. A fit-gap analysis will be performed. The fit-gap analysis 
will identify skill sets that require greater or less emphasis in e-business academic programs. The findings of this 
research will also help us to rethink and restructure the e-business programs to meet new challenges. 



INTRODUCTION 

The Internet has indeed transformed the world. E- 
Business has impacted almost every segment of the 
world economy. The plethora of opportunities opened 
in this cyber-frontier has taken the spirit of 
entrepreneurship to dizzy heights in the last decade. The 
Achilles’ heel in this new economy is the lack of IT 
professionals with the right skill sets — constant 
complaint from recruiters. This study investigates the 
alignment of e-business academic programs with the e- 
business job market. 

LITERATURE REVIEW 

E-business is defined as the conduct of business with the 
assistance of telecommunications and telecommuni- 
cation-based tools (Clarke 1 999). This definition covers 
a broad range of activities, including business-to- 
business (B2B), to business-to-customer (B2C), to intra- 
organizational commerce (Siau & Davis, 2001). The e- 
business “big-bang” caught most of the academic 
institutions by surprise. Not surprisingly, most schools 
were not able to follow suit with the requisite programs 



or courses. A discernible upsurge in e-business 
programs to overcome this lag is clearly evident, 
especially in the top US business schools. However, the 
nature of the response varies considerably among 
schools. Some schools have rejected outright the need 
for a separate e-business program, whereas other schools 
have embraced and staked their reputation on their new 
e-business programs. Are these programs producing the 
right kind of graduates for the industry? What are the 
core and optional courses in e-business programs? 

RESEARCH METHODOLOGY 

This study analyzes the e-business programs in the top 
business schools ranked by the US News and Business 
Week. We document the length and breadth of the 
various types of e-business programs offered by the top 
business schools through content analysis of their e- 
business program websites. In addition, we investigate 
the skill sets sought after by employers through content 
analysis of job placement websites. We will attempt to 
map the skills required by the employers for different 
jobs to the courses offered in e-business programs. This 
fit-gap analysis will shed light on the core and optional 
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courses in e-business programs and the skill sets needed 
by IT professionals in this e-business era. 

Data Collection 

The first task in this study is to analyze the business 
schools in the US to understand the length and breadth 
of the various types of e-business programs/courses they 
offer. Rather than doing an exhaustive search of all the 
e-business programs in business schools, which is 
impossible, we decided to look at the e-business 
programs in top US business schools. The top thirty 
business schools, ranked independently by US News and 
Business Week (see Appendix A), were selected as our 
sample. Since the two lists do not overlap completely, 
we have a total of 42 schools in our sample. The US 
News * ranking is for the top undergraduate business 
programs and the Business Week's ranking is for the top 
MBA programs. This sample, in our judgment, is 
representative of the past trends and the future 
projections in the e-business educational arena. These 
are the top business schools in the nation and they set the 
standard for education. It should be noted that the two 
lists provide us a mean to select our sample; but when 
performing data collection we look for undergraduate, 
graduate, and certification e-business programs in each 
school — irrespective of the list the school is in. 

The second aspect of our study will be to assess industry 
demand for e-business professionals and the skill sets 
needed for specific e-business career tracks. To perform 
this analysis, we will conduct an extensive content 
analysis of e-business job listings on major US Web job 
sites (e.g., Dice.com, Monster.com, and BrassRing.com). 
Specifically, the content analysis will focus on two 
outcomes: i) identifying general categories of e-business 
professions (e.g., Web site developer, Java programmer, 
data base specialist), as well as the relative percentage of 
positions within each profession; and ii) the specific job 
skills required for each position type. 

ANALYSIS AND PRELIMINARY RESULTS 

We have finished analyzing the websites of e-business 
programs offered by top business schools. The content 
analysis of major US Web jobsites is still ongoing. 



Types of E-Business Programs 

The content analysis of the top business schools’ 
websites reveals six types of e-business programs (as 
shown in Appendix B). Most of the existing programs 
are at the graduate level. Some programs are more 
focused (major or concentration) whilst others are 
subcategories of existing programs (e.g., minor). We 
also observed that some schools are beginning to offer 
certification programs in e-business. These are typically 
shorter programs and the aim is to train existing IT 
professionals in e-business. The duration of the MBA 
programs varied from 1-2 years, while the certification 
programs are less than one year. 

Based on the content analysis, we came up with 
Appendix C that categorizes the courses offered in these 
e-business programs. A detailed analysis of the courses 
offered by these schools showed that there are two broad 
and distinct tracks in the MBA programs. One track 
paves the path for an e-business career with a 
predominant focus towards the business aspect, whereas 
the other track leads to a career emphasizing technology 
and web-based systems development. 

Business track. Within the superset of e-business, this 
track consists of core courses in Law, Marketing, 
Finance/ Accounting, and Infrastructure and Technology 
for the Internet. The electives are geared towards 
shaping a career for the students in the areas of E- 
Business Analyst, E-Merchandising Consultant, E- 
Business Financial Analyst, Cross-Functional Enterprise 
Architect, and E-Commerce Entrepreneur. Also, a 
practicum is usually included to expose students to the 
industry. 

Technology track. This track typically starts with 
prerequisites/core courses in basic programming (e.g., 
C-H*/Java), E-Business Systems Analysis and Design, 
Database Systems Design and Implementation, and Web 
Development. The specializations are geared towards 
entering into technical positions in industry such as Web 
Developer, Technical Consultant, Systems Integration 
Specialist, Web-based ERP Analyst, Wireless Telecom- 
munications Analyst, XML Developer, E-business 
Infrastructure and Technical Architect, and E-Solutions 
Entrepreneur. 
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Initial Observations on the Fit-Gap 

Preliminary results of our fit-gap analysis provide 
several interesting findings: i) There is a growing 
demand in industry for ERP-Web integration. However, 
few, if any academic programs have addressed this 
particular issue; ii) Only one or two programs have 
focused on vertical industry specializations (e.g., 
financial services, health care) in the e-business 
curriculum; iii) Also, even though mobile commerce is 
growing in importance (Siau et al. 2001), few e-business 
programs address the issue of managing e-business for 
mobile users. Potential issues here include: development 
of mobile EB interfaces, creating and implementing new 
technologies for mobile computing, and security and 
reliability of mobile EB applications; iv) Managing e- 
business in an increasingly global environment is also 
noteworthy. Issues here focus on managing cross-border 
commerce: delivery of products and services, 

international monetary issues, governmental regulations, 
and consumer cultural differences; and v) Few academic 
programs offer courses in e-business architecture, even 
though many positions rely heavily on knowledge of 
how to set up and maintain the physical and application 
infrastructure for an e-business. 

CONCLUSION 

Our content analysis of e-business academic programs 
and e-business job advertisements will allow us to 



perform a fit-gap analysis by comparing skill sets 
demanded in business with skills taught in the major e- 
business programs. This study will permit us to begin to 
identify areas that require greater or less emphasis as 
market demands continue to evolve. By identifying these 
and other potential issues, we hope to begin to provide 
useful suggestions with regard to rethinking and 
restructuring e-business programs and, perhaps, helping 
them become more relevant to current business needs. 

This study is ongoing and will be completed by 
November. Full research findings will be available for 
IAIM’2001 presentation. 
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APPENDIX A 

US BUSINESS SCHOOL RANKINGS 



Rank 


US News 


Business Week 


1 


University of Pennsylvania 


University of Pennsylvania (Wharton) 


2 


Massachusetts Institute of Technology 
University of Michigan-Ann Arbor 


Northwestern (Kellogg) 


3 




Harvard 


4 


University of Califomia-Berkeley 


MIT (Sloan) 


5 


Carnegie Mellon University 
University of North Carol ina-Chapel Hill 
University of Texas-Austin 


Duke (Fuqua) 


6 




Michigan 


7 




Columbia 


8 


New York University 
University of Virginia 


Cornell (Johnson) 


9 




Virginia (Darden) 


10 


Indiana University-Bloomington 
University of Illinois-Urbana Champaign 
University of Wisconsin-Madison 


Chicago 


11 




Stanford 


12 




UCLA (Andersen) 


13 


Purdue University-Lafayette 
University of Minnesota-Twin Cities 
University of Southern California 


NYU (Stem) 


14 




Carnegie Mellon 


15 




UN C- Chapel Hill 


16 


Emory University 

Ohio State University-Columbus 

Washington University in St. Louis 


Dartmouth (Tick) 


17 




Texas-Austin (McCombs) 


18 




UC Berkeley (Haas) 


19 


Michigan State University 

Pennsylvania State University-University Park 


Yale 


20 




Indiana 


21 


Babson College 

Case Western Reserve University 

University of Arizona 

University of Florida 

University of Mary land-Col lege Park 

University of Notre Dame 

University of Washington 


Rochester (Simon) 


22 




Vanderbilt (Owen) 


23 




Washington University (Olin) 


24 




USC (Marshall) 


25 




Purdue (Krannert) 


26 




Georgetown (McDonough) 


27 




Maryland (Smith) 


' 28 


Arizona State University 

Georgetown University 

Texas A&M University-College Station 

University of Georgia 

University of Iowa 

Wake Forest University 


Emory (Goizueta) 


29 




Michigan State (Broad) 


30 




Georgia Tech (DuPree) 



(Source: US News Undergraduate Business Program Ranking 2001 & Business Week’s MBA Ranking 2001) 
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APPENDIX B 

TYPES OF E-BUSINESS PROGRAMS 




APPENDIX C 

CATEGORIES OF E-BUSINESS COURSES OFFERED IN BUSINESS SCHOOLS 



Introduction to E-Business 

• Electronic commerce 

• Infrastructure for e-business 

• Digital economy and commerce 

• Electronic commerce and virtual organizations 

E-Business Management 

• Project management 

• Management of technology and EC projects 

• Service process management 

• Managing the digital business 

• Management of small business systems 

• Managing information in the competitive environment 

E-Business Technology 

• Technology for electronic commerce 

• Operations infrastructure for the e-corporation 

• Emerging technologies 

• Networking technology 

• Advanced information technology 

• Business models and technology 



E-Business Strategy 

• Industry structure and competitive strategy 

• Managing inter-firm strategic alliances 

• Telecommunications technology and competitive strategy 

• Corporate strategies and practices in services industries 

• Financial strategy 

• Strategic issues in electronic commerce 

• Strategic management of technology and innovation 

• Strategic analysis for high tech industries 

E-Business Marketing 

• Entrepreneurial marketing 

• Electronic commerce and marketing 

• Marketing high tech products 

• Electronic channels 

• Consumer marketing 

• Advertising management 

• Customer experience design for E-business 

• Marketing research 

• New product development 

• Marketing decision support 
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Java and Object-Oriented Programming 

• Object-oriented and visual paradigms 

• Core java for electronic commerce 

• Programming in C and Java 

E-Business Economics and Markets 

• Managerial economics and game theory 

• Technology in global markets 

• Economics for managing information in electronic commerce 

• Economics for innovation and entrepreneurship 

• Markets and netcentric systems 

• Impact of the Internet on mature industries 

E-Business Systems Development 

• Development of e-business applications 

• Information retrieval system design 

• EC implementation 

• Development technology for the Web 

Operations and Supply Chain Management 

• Supply chain management 

• Operations and marketing integration 

• Manufacturing and Quality 

• Operations strategy and project management 

Legal and Regulatory Issues in E-Business 

• Legal, audit, and tax issues for e-business 

• Electronic commerce law and regulation 

Data Management 

• Data base management 

• Data mining techniques for business 

• Applied Data Analysis 

• Data management and decision support systems 

• Knowledge management 

Global E-Business Management 

• Technology in global markets 

• Global information systems 

• Global information technology management 

Business Process Reengineering in E-Business 

• Strategic business transformation 

• Managing organizational change 

• Managing information-intensive change 

• Leadership and change in organizations 

• Business process reengineering through information 
technology 



Data Communications 

• Communications technology and policy 

• Computer mediated communication and electronic commerce 

• Competition in telecommunications 

• Wireless network & mobility 

• Distributed component architecture 

• Business data communications 

E-Business Security 

• Computer security 

• Risk and controls in electronic commerce 

• Web commerce security 

Entrepreneurship in E-Business 

• Innovation, change, and entrepreneurship 

• High tech entrepreneurship 

• Entrepreneurial marketing 

• New venture creation ^ ' 

• Entrepreneurial finance 

Business Models in E-Business 

• Marketing models 

• Enterprise integration models 

• Internet business models and technology 

Electronic Payment Systems 

• Electronic payment systems 

Interface Design 

• Human factors in IS 

• User interface design for the Web 

• Advanced computer graphics 

• Multimedia systems 

Enterprise Resource Planning 

• Enterprise resource planning 

E-Business Practicum 

• Entrepreneurial laboratory 

• E-Commerce Project 

Computer Ethics 

• Computers, ethics, and society 

Intelligent Systems 

• Intelligent systems 
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ABSTRACT ' 

While this proposal gives a background for the development of the Criteria for Accreditation of Information Systems 
Programs, the panel will concentrate on the Criteria, preparing for an accreditation visit, and new developments in 
IS accreditation . 

Program accreditation has been a very effective m echanism for assisting in quality assurance in programs in many 
disciplines, particularly at the baccalaureate level. Computer Science programs have been accredited by the 
Computing Sciences Accreditation Board (CSAB) beginning in 1986. Now over 160 computer science programs are 
accredited and more programs seek accreditation each year [1]. Since its inception, CSAB has been interested in 
expanding program accreditation beyond Computer Science. With funding from a grant from the National Science 
Foundation (NSF), Criteria have been developed for accreditation of programs in Information Systems. These 
Criteria were approved CSAB in July 2000 and pilot accreditation visits were appro ved for the Fall of 2001, thereby 
allowing the possibility of accreditation of IS programs to begin in July 2002. 



INTRODUCTION 

With the rather recent developments in IS curricula, a 
good basis for the establishment of IS criteria for 
accreditation became available. The recent major 
activities in the development and dissemination of 
Curricula [Information Systems ’97 (IS’97), Information 
Systems Centric Curriculum ’99 (ISCC’99) [2] and the 
substantial effort on IS 2000] and the adoption or 
adaptation of these curricula are major contributions to 
the curricular portion of the accreditation criteria. 

Using the basic form and format developed by the 
Computer Science Accreditation Commission for 



accrediting Computer Scienceprograms (www.csab.org) 
as a pattern, the Criteria for Accreditation of Information 
Systems Programs were developed. The significant 
curriculum docum ents referenced above provided a basis 
for the Curriculum portion of the Criteria. The Draft 
Information Systems Accreditation Criteria were 
discussed at many conferences, including ACM 
SIGCSE, FIE, AIS, ISECON, IAIM and ICIS. 
Comments from these conference presentations were 
considered and the Draft Criteria were revised. The 
curriculum area seemed to bethe area of greatest interest 
and a web survey was conducted allow public review of 
the curriculum. Over 300 people responded to the web 
survey. The survey was analyzed and the Draft Criteria 
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were revised to become the proposed “Criteria for 
Accreditation of Informatio n Systems Programs.” 

WHAT CONSTITUTES 
AN ACCREDITED PROGRAM 

An accredited program is one that has beenreviewed by 
qualified and trained visitors and meets the Criteria 
established and approved by the CS AB. Only programs 
that are in institutions that have regional accreditation 
can be considered. This means that the evaluators 
visitingthe institution are assured thatthe institution has 
in place the necessary infrastructure to offer programs 
leading to the degrees they offer, particularly on the 
undergraduate (baccalaureate) level. Accreditation is 
voluntary and an institution wishing to be accredited 
must request an accreditation evaluation. 

WHO BENEFITS FROM ACCREDITATION 

Many groups benefit from accreditation, including 
students, parents, employers, the public, the institution, 
department and the program . Students benefit, by being 
assured that they are being taught what they need to 
know to prepare them to function as a professional in the 
information systems field. They can choose to attend a 
university that has shown that it provides the type of 
education deemed necessary by CSAB and the 
professional societies that have developed the Criteria: 
Association of Computing Machinery (ACM), 
Association for Information Systems (AIS), Association 
of Information Technology Professionals (AITP), and 
the IEEE-Computer Society (IEEE-CS). Parents have 
increasedassurance thatthey are paying foran education 
that will adequately prepare their son or daughter to 
enter the workforce or go on for graduate education in 
the field. 

Employers value accreditation because students 
graduating from an accredited program have attained a 
certain level of competence. T his provides the employer 
with a workforce that has acquired a certain level of 
ability in the field. CSAB enc ourages th eir employees to 
participate in the accreditation process, specifically by 
commenting on the Criteria, proposing andcommenting 
on changes to the Criteria, and serving as Program 
Evaluators and Team Chairs for site visits to evaluate 
programs. 

The public has a stake in accreditation, because as 
business, industry, government and individuals rely 



more and more on computers and the software that runs 
them, it is vital that minimum standards of preparation of 
workers be set in place. It is necessary that the systems 
upon which the public relies be designed, developed and 
maintained by competent professionals. Accreditation is 
one way to assure, at least minimum preparation, of 
people entering the field. As more and more computer 
systems can impact the life, health and safety of people 
everywhere, having competent information systems 
personnel is essential. 

Accreditation benefits the in stitution, the department and 
the program being accredited. The self-study is in and of 
itself of benefit to the program. The reputation of the 
institution is enhanced when the program is recognized 
as having met the standards of accreditation. The 
department benefits by having an accredited program 
when it recruits students and as it works to hire and 
retain faculty. Program accreditation assists in quality 
assurance. 

EXCERPTS FROM THE CRITERIA 
FOR ACCREDITING INFORMATION 
SYSTEMS PROGRAMS 

The criteria are divided into seven major categories. 
Each program must meet every intent. 

An Intent provides the underlying principles 
associated with a category. For a program to be 
accreditable it must meet the Intentstatement of 
every category. 

Objectives and Assessments 

Intent. The program has documented educational 
objectives that are consistent with the mission of the 
institution. The program has in place processes to 
regularly assess its progress against its objectives and 
uses the results of the assessments to identify program 
improvements and to modify the program’s objectives. 

Students 

Intent. Students can complete the program in a 
reasonable amount of time. Students have ample 
opportunity to interact with their instructors and are 
offered timely guidance and advice about theprogram’s 
requirements and their career alternatives. Students who 
graduate the program meet all program requirements. 
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Faculty 

Intent. Faculty members are current and active in the 
discipline and have the necessary technical breadth and 
depth to supporta moderninformation systems program. 

Curriculum 

Intent. The curriculum combines professional 
requirements with general education requirements and 
electives to prepare students for a professional career in 
the information systems field, for further study in 
information systems, and for functioning in modern 
society. The professional requirements include coverage 
of basic and advanced topics in information systems as 
well as an emphasis on a supportive area. Curricula are 
consistent with widely recognized mode Is and standards. 

Technology Infrastructure 

Intent. Computer resources are available, accessible, and 
adequately supported to enable students to complete 
their course work and to support faculty teaching needs 
and scholarly activity. 

Institutional Support and Financial Resources 

Intent. The institution’s support forthe program and the 
financial resources available to the program are 
sufficient to provide an environment in which the 
program can achieve its objectives. Support and 
resources are sufficient to provide assurance that an 
accreditedprogram will retain its strength throughout the 
period of accreditation. 

Program Delivery 

Intent. There are enough faculty members to cover the 
curriculum reasonably and to allow an appropriate mix 
of teaching and scholarly activity. 



Institutional Facilities 

Intent. Institutional facilities including thelibrary, other 
electronic information retrieval systems, computer 
networks, classrooms, and offices are adequate to 
support the objectives of the program . 

CONCLUSION 

With the recently developed, disseminated and adopted 
curricula in IS, the desire of CSAB to look beyond the 
accreditation of Computer Science and to consider 
accreditation of Information Systems programs, and 
funding from the National Science Foundation, 
everything seems to be coming together to make 
accreditation of Information Systems Programs a reality. 
The proposed “Criteria for Accreditation of Information 
Systems Programs” has been approved by CSAB and 
pilot visits are underway. It is anticipated that many 
programs will apply for IS accreditation in the near 
future. 
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ABSTRACT 

The challenges of teaching an introductory MIS course have changed over the last decade . In this action research 
project, a three phase approach was taken. Firstly, the problem was diagnosed and a strategy formulated. Secondly, 
action was taken to implement the strategy over one semester; and thirdly, in the reflective learning phase, the 
outcomes were determined and the strategy fine-tuned. The five challenges faced by introductory MIS course 
designers were identified as the increased emphasis on flexible delivery, technology-smarter students, students 
expectations of multimedia teaching approaches, lack of technology and faculty resources, and finally, the dynamic 
nature of the course content. Examination of these issues resulted in a strategy involving partnerships, assessments, 
teams and technology (PATT). This PATT strategy was implemented to overcome the identified challenges. Early 
outcomes of the programme indicate the strategy is successful in terms of student attendance at tutorials, student 
performance, and faculty satisfaction. 



INTRODUCTION 

The University of Southern Queensland (USQ) has built 
a reputation since its founding in 1967 for offering 
quality academic courses which are recognised 
worldwide, and in particular for the excellence of its 
distance education courses. As emphasised in its 
Mission Statement, USQ provides access for learners 
from diverse social, economic and cultural backgrounds 
worldwide, whilst recognizing its pivotal role in the life 
of its regional communities. USQ’s commitment to 
choice in the modes of delivery of its programs on- 
campus, off-campus and online, within a people-centred 
modem communications environment, identifies its 
distinctive contribution (USQ 2001). USQ is located at 
Toowoomba in southeast Queensland, Australia. 



Approximately 5000 students, including about 900 
international students from 40 nations, study on-campus, 
and 13,000 off-campus by distance learning including 
3,000 off-shore students (USQ 2000). 

USQ graduates are strongly sought after in the 
workplace, and independent surveys show they have a 
very high overall satisfaction rate with the standard of 
the courses and support offered to them. The University 
is truly student-centred and emphasises learning 
experiences and individual needs. This case study reports 
on the approach taken by course designers at USQ to 
improve the Introductory MIS course offered by the 
Faculty of Business and Commerce. 
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The goals of this study were to identify challenges in 
teaching on-campus introductory MIS courses in a 
regional university; to identify a strategy to overcome 
these challenges; and to evaluate the outcomes from 
implementing the identified strategy. 

Measurable Objectives 

To evaluate the outcomes from implementing the 
strategy, the following measurable objectives were 
formulated. The strategy should result in: 

• An increase of 10% in the number of students who 
pass the course without dropping the pass cut-off mark 
below 50%; 

• An increase of 10% in the number of students who 
attend lectures and tutorials; 

• An increase in student knowledge of unit content 
measured through a comparison between students’ 
perceived level of knowledge prior to enrolment and 
after completion of course (see Appendix A for a 
sample survey); 

• A perceived increase in the overall popularity of 
course and its teaching staff, measured through 
teaching evaluations of lecturer and tutor; 

• An increase in the number of research and scholarship 
goals achieved by faculty members. 

In the next section, the methodology is described, then 
the identified challenges are discussed. The strategy 
which was formulated is then detailed, and the outcomes 
presented with a discussion about future enhancements. 

METHODOLOGY 

The methodology used for this research is based on 
action research, combining theory and practice through 
change and reflection in an immediate problematic 
situation (Avison, Lau, Myers & Nielsen 1999). 

The research consists of three phases. Phase 1 diagnosed 
the problem, determined the objectives and then 
formulated the strategy. In this phase, a brainstorming 
session was conducted with faculty involved in teaching 
and developing foundation courses. Then the value 
chain analysis model was applied to identify specific 
critical leverage points where the Introductory MIS 
course could be enhanced (Laudon & Laudon 2000). 



Factors within the tertiary education sector as well as 
specific to regional universities were considered (Potter 
1985). This analysis process resulted in a 4-faceted 
strategy to overcome the challenges of teaching an 
introductory MIS course. 

In the second phase, action intervention occurred as the 
strategy was implemented. The student cohort was 
surveyed to collect and analyse their demographics and 
their perceived skill level and attitudes towards the 
foundation MIS course. Then the strategy was 
implemented by changing the content and delivery of 
material to on-campus students. 

The third phase of the study involved reflective learning 
as the outcomes were determined and the strategy 
reviewed and improved. In addition, continued 
observation for a prolonged period (longitudinal-based) 
would further confirm the applicability and effectiveness 
of the strategy. 

BACKGROUND 

The University of Southern Queensland has placed 
Foundation Computing as one of its main strategic 
educational objectives for over a decade. The 
Foundation Computing course — Introduction to 
Computing — has a long history of educating first year 
tertiary students at the University of Southern 
Queensland. Table 1 shows the number of students 
enrolled in the course in 1990, 1995 and last year. For 
some years, the course was offered as a core subject to 
all USQ undergraduate students, but now other Faculties 
offer their own course. 

In a survey conducted at the start of the 2001 academic 
year the following demographic information (Table 2) 
highlighted characteristics of the students studying the 
Introduction to Computing course. In Queensland, 
children are aged five years when they commence 12 
years of formal schooling. Therefore, most high school 
students who go on to commence university study 
immediately after school are aged 17 years. However, 
the average age of students who were enrolled in this 
first year course is 20.5 years. This is indicative of the 
trend of mature-age people undertaking university 
studies as a result of pressure on individuals to reskill. 

CHALLENGES 

Faculty staff involved in the Foundation MIS course 
participated in a brain-storming session which was held 
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TABLE 1 

COURSE FOCUS AND ENROLMENTS 



1990 


1995 


2001 






Tim o 






x i me 


• University-wide corporate unit 
(cross- faculty) 


• Faculties of Business/Commerce 
unit 


• Faculty of Business & Commerce 
(merged) 


• General computing studies 


• Option for other students 


• Option for other students 


• Course Enrolments all modes 
(approx.) 

• Semester 1 - 800 

• Semester 2 - 400 

• Semester 3 - 200 


• Practical Unit 

• Some emphasis on information 
systems 

• Course enrolments all modes 
(approx.) 

• Semester 1 - 400 

• Semester 2 - 200 

• Semester 3-100 


• IS unit with adequate emphasis on 
IS technical skills 

• Course enrolments all modes 
(approx.) 

• Semester 1 - 650 

• Semester 2 - 450 

• Semester 3 - 200 



TABLE 2 

CHARACTERISTICS OF STUDENTS ENROLLED SEMESTER 1 2001 



Number of Respondents 


152 (67%) 


Age 


Mean 20.52 years 
Standard Deviation 5.83 years 


Gender breakdown 


53.3% males 
46.7% females 


Previous computing 
experience 


75% of respondents studied computing in high school 
77% of males studied computing in a formal setting 
72% of females studied computing in a formal setting 

None of the students over the age of 33 years had studied computing in a formal 
setting 



in late 1999. It was generally realised that the course 
needed to be upgraded and that a new approach was 
warranted. The brain-storming session identified the 
following challenges. 

Flexible Delivery 

Increased emphasis on flexible delivery has made it ever 
more challenging to structure an introductory MIS 
course in alignment with the needs of students both on- 
and off-campus. With the emphasis on lifelong learning, 
many students are mature-aged, bringing with them 
industry skills and technical experience. Flexible 



delivery incorporates a range of learning strategies (case, 
hands-on), learning environments (team, one-on-one, 
tutorial), learning styles (visual, audio, written), learning 
interests and needs (fee-based, cross-institutional, career 
goals, educational goals), and learning opportunities 
(geographical constraints, cultural climate) (OLIATAFE 
2001). At USQ, flexible delivery is achieved through 
on-campus, external (print-based) and online modes. To 
cope with these demands, faculty need to be equipped 
with a diverse range of skills and a variety of 
technologies need to be utilised to facilitate effective 
communication with students. 
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Student Cohorts 

The second challenge stems from the trend of high 
school students enrolling in pre-tertiary computer 
studies. Consequently, entry-level university students 
are ‘technology smarter’ than their predecessors. These 
predominantly on-campus students perceive themselves 
as having all of the skills and knowledge necessary to 
complete the course, without attending on-campus 
teaching. This over-confidence and complacency often 
results in poor attendance and performance. 

Traditional Teaching 

Characteristics of the MTV generation contribute to the 
third challenge: students’ failure to respond to traditional 
“chalk and talk” approach to lectures and tutorials. In a 
core MIS course, students expect that cutting-edge 
technologies will be showcased. They want to be 

entertained with all the ‘bells and whistles’ of 
technology. 

Industry Attractions 

The fourth challenge is experienced by most IT 
departments in tertiary institutions: lack of resources. 
One problem is in attracting and retaining staff with 
current skills. This is because academic remuneration is 
substandard to industry norms and there is tremendous 
demand for IT professionals. IT graduates often earn 
more than their lecturers. Currently, there are more than 
20 vacancies for information systems faculty in 
Australian universities. This unprecedented demand has 
resulted from a misalignment between industry and 
academic remuneration. In addition, there is a marked 
shortage of skilled information systems professionals in 
the industry. A recent survey of Australian businesses 
conducted by the IT Skills Hub through the Australian 
Bureau of Statistics, found that there will be an estimated 
24,000 additional people required to work in core 
Information and Communications Technology 
occupations and supporting roles in 2001 and a further 
27,500 in 2002 (Alston 2001). This coupled with ever 
increasing global demand for IS professionals has 
created a crisis in attracting and retaining suitable 
faculty. 

Universities are under increasing funding pressure to 
contract out course development and support materials. 
These strategies are often beyond the means of smaller 
universities. In addition, there is a need to equip course 
developers with state-of-the-art hardware, software and 



data communications equipment. To utilise state-of-the- 
art technology, campus laboratories must also be re- 
equipped with necessary hardware and software. All of 
these requirements put further strain on the faculty 
budget. 

Content Dynamism 

The fifth and final challenge identified relates to the 
ever-increasing rate of technology change which forces 
continual updating of study materials. New increments 
of software packages continually lead to new text and 
materials. These need to be integrated into the existing 
course. Course developers need to continually update 
their knowledge and skills to keep pace with 
technological changes. A diverse and complicated skill 
set is required of course developers, including hardware, 
operating system, multimedia, web development, e- 
commerce, social and ethical issues, networking and 
communications expertise. 

PATT STRATEGY 

To overcome these challenges, an innovative approach 
has been developed and trialled with measurable success. 
This strategy is a multi-faceted strategy, consisting of 
four elements: student cohort skills assessment, teaching 
approach, publisher partnerships, and use of Internet 
technologies. The action research method is structured 
according to Partnerships, Assessments, Teams and 
Technology — The PATT Strategy, as it has been termed 
by the researchers and shown in Figure 1. The 
derivation process of identifying the PATT Strategy is 
intricately interwoven in theory and practice — quite 
appropriate for action research as action research 
literature was drawn upon, allowing the researchers to 
react to the contextual needs of research and practice. 

Partnerships 

Collaboration with publishers of adopted textbooks was 
leveraged to provide access to state-of-the-art 
information and up-to-date textbooks. Feedback to the 
authors of textbooks, maintained through the book 
publisher, allowed for continual content enhancements 
to counter textbook deficiencies. This feedback channel 
also offered the facility for lecturers to request 
additional text supplements specific to the course 
objectives. 

Publishers have begun to promote the mix-and-match of 
chapters and/or sections from various textbooks. This 



Proceedings of the 16 th Annual Conference of the International Academy for Information Management 



99 



FIGURE 1 
PATT STRATEGY 




Technology 



facility, known as Publishing on Demand, allows course 
designers to focus on course objectives as the basis for 
text selection — not the other way around. It also has the 
potential to reduce student expenses; when requiring 
parts from two books, one book can be published 
including both parts. 

Many publishers now offer teaching support with 
electronic resources such as instructor supplements, 
PowerPoint presentation slides, chapter summaries, 
answers to review questions, course Web sites, and video 
and image galleries. These useful add-ons were used to 
assist course developers by reducing staff workload and 
enhanced the learning experience for foundation MIS 
students. 

At the same time, publishers have also offered course 
designers remuneration to design supplements or 
enhance textbooks. The benefits are three-fold: 
academic recognition for course designers, financial 
rewards for course designers and ultimately better course 
materials. 

Assessment 

At the start of each semester, the skill and knowledge 
level of incoming students were assessed through the use 
of surveys (sample included in Appendix A). Results of 
the survey assisted the lecturer and teaching assistants 
(tutors) in customising teaching at an appropriate level 



based on student skill and experience. In conjunction 
with learning centres located within the university, a 
collaborative team was formed in which syllabi writers 
of the learning centres liaise with the course designers to 
align pre-university computing courses with foundation 
MIS courses. Alignment does not necessarily imply 
redundancy of content at the different levels, but 
building on to previous knowledge to supplement and 
support active learning at a higher level. 

The second area of assessment is pre-entry examinations. 
This is currently being actively debated as a means of 
providing exemptions for those who satisfy technology 
and scholarship criteria of the foundation MIS course. 
The intention would be that incoming students who 
perceive themselves to be competent in all aspects of the 
course be given the option to sit for an examination. To 
be granted an exemption to the course, students must 
pass both practical and theory-based questions, and 
achieve at least 65% overall in the examination. 

To maintain student interest in course content, flexibility 
in the choice of assessments is incorporated on the basis 
of student’s major study areas. Students are offered a 
range of assignments of various disciplines, including 
Visual Basic programming for computer programming 
majors, statistical analysis for economics and marketing 
majors, and; advanced spreadsheet skills for accounting 
and finance majors. In order to maintain some structure 
in the midst of incorporating flexibility, a single realistic 
business case study is carried through all assessments 
throughout the entire semester. These assignments are 
integrated to form a realistic business application, 
utilising various software packages such as word 
processors, spreadsheets, databases, presentation 
software and Web publishing, and combine these 
technical skills with a systems analysis and business 
information systems flavour. 

Team Teaching 

A team teaching approach ensured continuity of course 
development, supported collaborative teaching, and 
offered complementary skills. This is one way to reduce 
the “burn-out” syndrome experienced by teaching staff 
of entry-level MIS courses. It also allows teaching staff 
to strike a balance between teaching and other academic 
and scholarship pursuits. 

Traditionally, one team leader was responsible for all 
course design and management of the unit. As 
enrolments have grown and the challenges of 
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administering such a large course have escalated, the 
approach of utilising co-team leaders has been trialled 
with measurable success. The teaching duties of both 
the team leader and team moderator have been organised 
such that these roles are swapped on a semester-by- 
semester basis, allowing for a synergetic leadership of 
the course and its teaching assistants. This approach also 
offers continuity of content, teaching and continual 
alignment of teaching philosophy. At the same time, this 
allows the person taking on the role of moderator to 
pursue teaching in other areas. This approach also 
allows faculty to undertake recreation leave and/or other 



personal/research/career pursuits for a substantial dura- 
tion with no adverse effect on students. 

An interactive teaching and learning approach is 
undertaken in lectures and tutorials. These sessions 
utilise elements of technology and entertainment such as 
games (eg. The Weakest Link), debates on contemporary 
MIS issues, role-plays, student presentations in both 
learning and as an integral part of assessment, and real- 
life focused case studies. Table 3 illustrates a few 
activities undertaken. 



TABLE 3 

EXAMPLES OF ACTIVITIES UNDERTAKEN BY STUDY MODULE 



Study Modules 


Activities Undertaken 


Using Computers 


Debate on contemporary issues facilitated ice-breaking and group discussion. 
Students were asked to briefly research one of the following topics: 
telecommuting, electronic money, paperless office. 


Application Software 


Copies of magazine articles and promotional brochures were distributed to 
students. They were asked to classify these application software packages into 
appropriate categories, based on a checklist of features which was created in an 
in-group activity. 


The System Unit 


Students were presented with a variety of hardware components, including a 
workable computer. This allowed for a look-and-feel approach to identifying 
system components and understanding how these components interconnect and 
operate. 


The Internet 


Study of the Internet lent itself well to a brainstorming session of previous, 
current and future Internet-based applications and the impact of these on 
information systems and society as a whole. Demonstrations of various Internet 
services and WWW applications such as chat, messaging, plug-ins, 
videoconferencing were also incorporated in lectures to showcase contemporary 
innovations on the Web. 


Operating Systems 


A sketch was set up with groups of students personifying on the different 
functions of an operating system. Students were asked to ‘act-out' the process 
of memory management, spooling, performance management etc. 


Networking 


The network laboratory was booked for all workshop sessions to enable 
students to set up a simple Ethernet network. This assisted them in 
understanding the various connection types as well as exposed them to various 
network management software tools, physical transmission media and 
connectivity hardware. 


Mid-semester recap of content 


Final year IT project students designed a computer program to simulate the 
game show The Weakest Link. The program was incorporated into a mid- 
semester lecture session as a means for students to recap their knowledge of 
content covered in a fun, interactive environment. 
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These activities above and many others that were used in 
tutorial and practical workshops are all aimed at 
stimulating student curiosity to form the basis for active 
learning (Berlyne 1960). Perception curiosity 
(surprising or complex sensory stimulus) in the form of 
multimedia elements were incorporated in all lectures by 
use of CNN video clips and other sound/video clips 
related to computing and the paradox of computing. 
Epistemic curiosity (discrepant thoughts, beliefs and 
attitudes) was triggered through case study analysis of 
situations. Students were put in unfamiliar roles in 
unfamiliar environments e.g. role plays of interaction 
between components of the computer. This increased 
arousal and exploratory behaviour in students, reduced 
discrepancy of understanding and challenged students to 
look at all facets of a situation. As shown in Figure 2, 
this approach should help students develop the habits of 
active lifelong learning. 

FIGURE 2 

STEPS TO LIFELONG LEARNING 



Arouse 


\ 


Introduce 


\ 


Enhance 


— N 


Achieve 


curiosity 


/ 


motivation 


J 


performance 


— / 


results 




Active Lifelong Learning 



The faculty has also employed a number of final year IT 
students as proctors or student assistants to support the 
facilitation of teaching in practical workshops. Proctors 
assist the tutor in initiating and implementing practical 
workshops. These proctors have previously completed 
the course and achieved a more than satisfactory grade. 
The proctor’s duty statement includes: 

• familiarity with a range of software packages and 
computer laboratory facilities; 



Firstly, as an information dissemination tool, the Web 
was used as a means of encouraging pre-lecture reading 
and to increase lecture attendance by providing 
information pertaining to the study module and to the 
upcoming lecture. Reference materials were also 
published on the Web where they could be easily 
accessed and downloaded by students as and when 
required. With the inclusion of min-versions of lecture 
slides, students were given a sneak preview of the 
upcoming lecture and content to promote attendance at 
lectures. 

Secondly, Web messaging technologies were utilised to 
their full advantage. These included e-mail, mailing 
lists, discussion groups, and Internet chat facilities. E- 
mails formed the basis for scheduling student 
consultation; a fast efficient way of answering short, 
specific queries; and provided an effective method for 
course team to communicate with each other in relation 
to teaching issues. 

For information dissemination to external students in 
Australia and in other countries, mailing lists have been 
established. The chief advantage of course-specific 
mailing lists is to facilitate flexibility in course changes 
throughout the semester. Additions and/or modifications 
to course requirements can be easily communicated to all 
students by this method. 

Discussion groups were established as a means to 
promote and facilitate discussion between students. This 
method has demonstrated its effectiveness in assisting 
learning and also serves as an avenue for grievance and 
student support. 

Internet-based messaging and chat facilities are also 
starting to gain popularity in use. The benefits are two- 
fold: encouraging students to use contemporary 

technologies that they have learned in the course, and 
providing another means of feedback and 
communications loop between faculty and student. 



• understanding of assessment requirements; and 

• attendance at lectures and teaching team meetings. 

TECHNOLOGY 

One of the key technologies used in the case is the 
Internet. The Internet was used as an information 
dissemination tool, for messaging to students and also 
provided the capability for an online assessment engine. 



The third internet based technology involved the use of 
an online assessment engine which allowed students to 
self-evaluate their skills and knowledge. At the same 
time, lecturers were able to use statistics of access times 
and assessment results to track student participation 
levels and identify weak knowledge areas. Web online 
assessment systems provided by book publishers greatly 
reduced the workload of course designers in the 
preparation of quizzes and feedback sheets. Publishers 
provide server facilities for storage and management of 
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these quizzes as well as data on student participation in 
quizzes, student activity on the site and quiz results. 
These assessments can either be formative or summative, 
depending on course requirements. To date, the 
technology is still on trial and therefore, only formative 
assessments have been used. 

Issues that need to be considered when choosing an 
online Web-based assessment engine include cost to 
students and the university, server reliability and 
performance, ease of management and flexibility of 
assessment styles. With substantial student numbers, 
most book publishers offer the use of the engine at 
nominal or no cost to students and the university. The 
physical location of the engine is an important 
consideration. Some institutions choose to set up their 
own servers residing on-campus to host the assessment 
engine. Others prefer the benefits of outsourcing the 
maintenance costs and performance management issues 
to the book publishers. All assessment engines should 
contain a user-friendly interface for managing student 
data, student results, and administering assessment items. 

The content that can be assessed through an online 
assessment engine offered by book publishers is directly 
related to the adopted textbook. Other content is not 
usually permissible on the engine. Unfortunately, such 
online assessment engines tend to focus only on the 
theoretical components. For courses that assess both 
theory (computing concepts and principles) and practical 
components (technical word processing, spreadsheet and 
database skills) of their foundation MIS courses, other 
assessments need to be designed to test practical skills. 
Online assessments are popular with many students as 
the feedback is immediate and the tests can be taken at 
a time and location convenient to the student. This form 
of assessment provide initial extrinsic motivation which 
is advocated by Lepper (1988) to nurture intrinsic 
motivation to leam. 

FINDINGS 

To date, measurable positive impact has been achieved 
in implementing the strategies outlined above. Some 
quantitative measures include student retention at 
lectures and tutorials, movements between tutorials and 
pass rate. These are detailed in Table 4. 

Tutorial staff acknowledge that though the PATT 
strategy initially increased their workload in regard to 
materials development, they see a direct flow through 
benefit to themselves in regard to higher student 



attendance and participation. Testimonials of tutors 
include classes are running smoother, students are 
enjoying learning and I don’t feel as drained at the end 
of the day as I once did”; “... they want to know more, 
the multimedia elements are just whetting their 
appetites.” 

Website and news group usage monitoring further 
highlight the impact of the PATT strategy. Previous to 
the strategy adoption in semester 1, hits to the unit 
website averaged seventeen per week, currently unit 
website hits exceed 2,500 per week. Active participants 
in news group discussions numbered on average thirty- 
five prior to the PATT strategy adoption, currently fifty- 
five percent of external students and thirty percent of 
internal students are using this resource. 

Some qualitative feedback obtained through course 
evaluation surveys conducted at the end of the semester 
revealed that students were satisfied with the knowledge 
and skill building activities both in lectures and tutorials, 
and felt that the strategy had allowed them to build upon 
the performance in both theoretical and practical aspects 
of the course. 

In order to boost students’ interest and participation, the 
course designers deliberately attempted to include an 
element of fun in the lectures and tutorials. Although 
Gagne and Briggs (1979) warn that ‘fun’ learning may 
detract from students comprehending and analytically 
understand concepts, in this case, many students 
commented positively about this aspect: “(name)’s 
teaching made learning computing interesting, I got 
more out of the lectures and tutes than I ever did at high 
school.” ; “... made learning about computers fun, I was 
dreading this course, but now I’m considering doing 
more computing courses”. 

As with any major change in teaching approach and 
organisation, the supply and management of resources 
are major issues. There is a need for a champion sponsor 
to undertake the responsibility of ensuring resources are 
ample and logically allocated. In the case study, the 
Head of Department was an active participant at 
meetings, was actively making decisions with the course 
team, and developed some teaching methods to support 
the implementation of PATT. The PATT strategy also 
requires a leader to encourage a team of self-motivated, 
independent and diligent educators with the mindset of 
educating and supporting students in the first year of 
tertiary studies. 
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TABLE 4 

PATT OUTCOMES SEMESTER I 2001 



Evaluation Criteria 


Semester 1 2001 


Semester 1 2000 


Student retention at lectures 


Increase of 1 5% 


Control 


Student retention at tutorials 


Average; 80% attendance 


Average: 65% attendance 


Movement between tutorials 


Dynamic in the first two weeks of the semester 


Rigid tutorial placement 


Pass rate 


62% pass rate 


49% pass rate 



CONCLUSION 

1 

The strategy described in this paper provides practical 
guidance for introductory MIS course designers who are 
looking for cost-effective ways to develop and redevelop 
technology-intense course materials. Course designers 
are urged to make full use of support from publishers, 
and to include Internet tools to facilitate material 
availability and communication. This research 
concludes that the challenges of teaching to a diverse, 
technology-smart entry-level cohort can be overcome by 
understanding the characteristics of the students and 
building flexibility into course delivery and assessment. 
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APPENDIX A 

SAMPLE STUDENT SURVEY 



Personal Particulars 



1 . Are you predominantly 

G a day student G an external student G a USQOnline student 

2. Age years 

3. Gender 

G Male G Female 

High School Education 

4. Were you 

G private high school G state high school student 

5. Name of high school 

6. Did you do any computing studies in high school? 

G Yes G No 

If no, go to Q8. 

7. If yes: 

G junior high computer 

G senior high computer studies - Information Processing and Technology 
G senior high computer studies - Computer Studies (CS) 

G senior high computer studies - Business Communications and Technology 
G other 

Use of Computers 

8. Have you ever used a computer? 

G Yes G No 

If no, go to Question 13. 

9. When did you first use a computer? Year 

10. Do you have a computer at home? 

G Yes G No 

If no, go to Q12 

1 1 . If yes, when did you first get your first computer at home? Year 
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1 2. On average, how many hours per week do you spend on a computer? 
G 0-2 hours G 3-6 hours G 7-10 hours G more than 10 



13. Computing Skills 

Please rate the following statements based on your skill in using the following. 



Word processors 

Spreadsheets 

Database systems 

Presentation software 

Desktop publishing software 

Accounting and financial software 

Drawing packages 

E-mail 

Newsgroups 

Internet chat software 

Internet browser 

FTP 

Gopher 

Programming languages (e.g. 
COBOL, Pascal, Visual Basic, Java) 
Computer games 

Obtain and play music (e.g. MP3s) 



Limited 

expertise 




Moderate 

expertise 




Expert 


< 

z 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 
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14. Frequency of Use 

Please rate the following 'statements based on the frequency of use per week in the following computing 
skills. 



Word processors 

Spreadsheets 

Database systems 

Presentation software 

Desktop publishing software 

Accounting and financial software 

Drawing packages 

E-mail 

Newsgroups 

Internet chat software 

Internet browser 

FTP 

Gopher 

Programming languages (e.g. 
COBOL, Pascal, Visual Basic, Java) 
Computer games 

Obtain and play music (e.g. MP3s) 



3 

0 

x: 

cn 

1 


3 

0 

-C 

1 

m 


3 

o 

-C 

00 

m 


8-10 

hours 


Greatei 
than 1C 


(N/A) 

Never 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 
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15. Attitudes 



How would you rate your level of agreement with the following statements? 





Strongly 

disagree 






Agree 






Strongly 

Agree 


< 

2 


Computing skills are important to me. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Computers help me to learn. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


A computer is a useful tool for me. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


1 feel it is difficult to use a computer. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Computers are as important to 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


students as textbooks. 


Computers make better thinkers out 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


of students. 


I use the computer effectively. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


I think computers should be used 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


wherever possible. 


When 1 have a choice 1 prefer to use a 
computer for some tasks. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Educational curricula should be 
reorganized to make maximum use of 
computers. 


D 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Computers isolate students from one 
another. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


I enjoy using a computer. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Owning a computer is highly 
desirable. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


I object to the attention being given to 
computers. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


I object to the attention being given to 
the Internet. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 
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ABSTRACT 

This paper describes efforts at the Department of Informatics of University of Pretoria to teach the principles of data 
warehousing to graduate students. The aim of this project was to teach students the complexities of a real world data 
warehousing effort. Students, who worked in groups of two, were required to design andimplement adata warehouse 
using a data repository concerning data as collected in the Pretoria region of the Gauteng Province of South Africa, 
during the 1991 and 1996 Censuses. 

To ensure that the real world is simulated as well as possible, the quality of the final data warehouse was analysed 
by considering how well the user requirements were being met. The success of this teaching approach was assessed 
through questionnaires and informal interviews, to determine the level of cognition attained by the individual 
students. The resultant data warehouse will subsequently be used for a data mining effort, concerning the analysis 
of health issues and the occurrence of crime in this region. 



INTRODUCTION 

Data warehousing and its related technologies are 
increasingly considered a way of addressing the 
strategic needs of an organisation (Berson, 1998; Gray 
& Watson, 1996). Successful data warehouses support 
the decision support requirements of businesses by 
providing integrated, high-quality subject-oriented 
information that has been collected overtime. This is 
done through careful integration and aggregation of 
current and historical data from different operational 
systems (Inmon& Keller, 1994). The data warehouse’s 
focus is on the analysis of integrated operational data 



from various sources, differing considerably from 
operational databases’ goal of recording and completing 
business transactions, implying different database 
design techniques. 

Although data warehousing provides information to 
potentially support critical success factors, a large 
number of well-funded data warehouse projects fail 
nevertheless. Many factors contribute to the failure of 
data warehouse projects. According to Adelman (1998) 
and Gibbons ( 1 997), amongst others, the failure of main 
data warehousing efforts is largely due to unhappy users 
and the perceived poor performance of the system, 
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caused by unfounded user expectations. This opinion 
seems to be shared by many practitioners: in a survey of 
attendees of a Data Warehouse Institute conference, the 
most important critical success factors of a data 
warehousing effort were identified as upper 
management support and user involvement (Gray & 
Watson, 1998). Greenfield (in Mallach, 2000:584) 
emphasises user involvement by calling data 
warehousing a joint user/builder project and 
responsibility. It is thus clear that end-user involvement 
throughout the development process is generally 
considered as of crucial importance. 

The teaching of data warehousing usually takes place in 
a regulated andpredictable classroom environment. The 
curricula typically covers the theoretical aspects of data 
warehousing i.e., the data warehouse methodologies, 
data modelling, extraction, transformation and 
transportation, data management, access and reporting. 
The practical component involves an introduction to 
enabling technologies and possibly a practical project 

This paper describes a different approach to data 
warehousing teaching. The Department of Informatics 
of University of Pretoria (UP) offers a data warehouse 
course to graduate students, who are considered to be 
familiar with the fundamentals of data modelling, data 
integration and the necessary technologies. The course 
covers theoretical aspects of data warehousing as well as 
an introduction to the enabling technologies. The course 
also includes a data warehouse project, using a relational 
database management system (DBMS) such as Oracle or 
SQL Server. Since it is assumed that students have 
mastered the basics of data modelling, the focus of the 
course is more on exposing students to the problems 
surrounding real life development and implementation 
of data warehouse solutions, as described here. 

The paper is organized as follows. Section 2 briefly 
introduces a number of data warehousing methodologies 
and discusses the pros and cons of each technique. 
Section 3 provides a brief justification of the teaching 
and learning approach, implementing the idea of 
‘reflective practitioner’ first introduced by Schon 
(1987). The section also shows how the learning 
approach relates to Bloom’s taxonomy of educational 
objectives. Section 4 describes the data warehousing 
case study. In Section 5, results and feedback from 
students are also discussed. Finally, Section 6 concludes 
the paper. 

DATA WAREHOUSING METHODOLOGIES 



With the advent of data warehousing, organisations are 
investing huge amount in order to obtain information 
from their vast data repositories. The main aim of most 
data warehousing efforts is the expected ROI, due to the 
perceived improvement in decision-making capabilities. 

To this end, a number of data warehousing development 
methodologies have been proposed, ranging from the 
“big bang approach to the iterative, incremental 
approaches (Gray and Watson, 1998; Hammergreen, 
1996). 

In the big bang approach, the data warehousing project 
is attempted in one phase. That is, the development of 
the data warehouse is seen one huge task to be 
completed at once. This approach is well suited to an 
environment where it is completed as part of a business 
re-engineering exercise. However, the big bang 
approach is considered a high-risk approach, since user 
requirements and technologies could change before the 
project is completed. 

With the iterative, incremental approaches, the data 
warehouse is developed in increments, whilst complying 
with a full-scale data warehousing architecture. That is, 
an enterprise-wide “road map is provided, which are 
then used to incrementally develop data marts for only 
one or two subject areas. In this way, the user is 
incrementally provided with usable information, while 
the data warehouse designers are being provided with 
the possibility to adapt the functionality of the system 
after each increment. 

The top down iterative approach is mainly driven by the 
business benefits and is widely recommended as a 
method that is quick to implement. Here, the business 
requirements are determined and a conceptual 
information model is formed. This method thus focuses 
on the creation of a flexible data warehouse architecture 
that could withstand the pressures of dynamic 
environments with unpredictable uses. On the other 
hand, the bottom up iterative approach focuses on the 
data , which resides in operational databases, legacy 
systems and external sources. Instead, the relatively 
quick implementation and focus on high-level business 
needs of the top down approach and the ready at hand 
data sources for initial application development of the 
bottom up approach make a combination of the two 
approaches a viable option (Mallach, 2000). 

Whichever method is chosen, high user involvement and 
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analysis of enterprise requirements to identify and 
prioritise subject areas stay important (Mallach, 2000; 
Greenfield in Mallach, 2000). However, it is clear that 
this user-centred approach is not well suited for the 
classroom environment. Real world users are difficult to 
find, due to the high costs associated with building a 
data warehouse, the time consuming nature of this 
exercise, etc. This leads to the danger that scoping and 
the identification of requirements to use in the approach 
become only a theoretical exercise, with the students 
thus adopting a typical IT driven bottom up approach. 

Research shows clearly that although theoretical and 
technological know-how regarding data warehouse 
development are necessary conditions for success, the 
skills of working with end-users and interpreting user 
requirements are critical success factors. The 
involvement of users introduces a human element, 
which often results in unpredictable and uncertain 
situations. The teaching approach described in this 
paper, focuses on fostering those skills necessary to 
handle real-world situations of which end-users are very 
much part of. The following section provides a 
theoretical background for our chosen approach. 

EDUCATING REFLECTIVE DATA 
WAREHOUSE DEVELOPERS 

Arnold Schon first introduced the concept of the 
“reflective practitioner. The idea was bom from the 
concern that professionals are not ready for the demands 
of the real-world practice. Schon (1987) argues that the 
education of professionals is largely based on a 
positivist epistemology of practice, called technical 
rationality. Technical rationality holdsthat practitioners 
are instrumental problem solvers who select technical 
means best suited to particular purposes. The 
assumption is that “Rigorous professional practitioners 
solve well-formed instrumental problems by applying 
theory and technique derived from systematic, 
preferably scientific knowledge. (Schon, 1987:4). 
However, experience shows that problems in real-world 
practice present themselves as messy, indeterminate 
situations. The way in which we deal with these 
unstructured problems is by giving it structure through 
‘naming and framing’ and only then applying technical 
problem solving. 

Schon argues that we need to teach those competencies that 
enable the professional to make decisions under conditions 
of uncertainty, i.e., to prepare the professional for the 
uncertain zones of practice. The argument is not against 



scientific knowledge, but for the recognition of the 
importance of the competencies displayed in uncertain and 
conflicted situations of practice. Schon considers the 
‘wisdom’, ‘intuition’ or ‘talent’ that enableprofessionalsto 
handle indeterminate zones of practice, as professional 
artistry. 

The question then arises whether it is possible to teach 
“professional artistry. To find an answer to this 
question, Schon directs his attention to the teaching of 
the arts: the dance school, the music class, the 
architectural design studio etc. Here he finds coaching 
instead of teaching and above all learning by doing. 
Schon describes the activities perceived in these settings 
as reflection-in-action. The teacher provides an 
uncertain situation (for both the teacher and student) and 
engages with the student in dialogue in the form of 
reciprocal reflection-in-action. Reflection is induced by 
constant surprises as a result of the uncertain situation. 
Students (and teachers) are forced to reflect on previous 
and future actions. Through reflection, the student 
adjusts his/her actions and knowing about the subject 
matter that eventually lead to wisdom and the 
refinement of intuition. 

Reflection-in-Action and Bloom’s Taxonomy 

It is the opinions of the authors that the simulation of a 
real world scenario provides students with an 
understanding of not only data warehouse concepts but 
also the difficulties surrounding data warehouse success. 
We believe that the success of the ‘reflective 
practitioner’ approach lies in the fact that all levels of 
Bloom’ taxonomy of educational objectives are being 
addressed and assessed. 

According to Bloom’s taxonomy, students need to be 
able to have knowledge and comprehension of the 
subject matter, be able to apply the knowledge, analyse 
and synthesise the concepts and lastly to be able to 
evaluate the concepts and methods on both a qualitative 
and quantitative level [Matthee, 1998]. Bloom 
distinguishes between six levels of knowledge (in order 
of complexity), namely remembering of facts, 
understanding, application, analysis, synthesis and 
evaluation. Van Loggerenberg ( 1 995) showed that there 
is a relationship between Bloom’s taxonomy and data, 
information, knowledge and wisdom or competence [De 
Villiers, 1996]. According to De Villiers (1996), for 
learning to take place, the lecturer should be able to 
convert her knowledge of information back into data, to 
enable the student to become competent in the subject 
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matter. This process is depicted in Figure 1 . The highest 
level of cognition, according to Bloom (1956), is thus 
evaluation, which is achieved when the student is able to 
apply his knowledge in an experienced matter and is 
able to reason about possible solutions. This level of 
cognition is seldom reached in the traditional teaching 
of data warehousing concepts, since students usually do 
not receive sufficient feedback (from the user) regarding 
their projects. 

The aim of the data warehousing project is therefore to 
create such a setting, i.e., a setting that forces reflection- 
in-action. This is done by simulating a real-world data 
warehouse development scenario, through the 
involvement of real life users in the form of the 
department’s data mining research group, which are 
actively involved throughout the development process. 
The next section describes the data warehousing case 
study and the user requirements in more detaiL 



DATA WAREHOUSING CASE STUDY 

A data warehouse is usually constructed from a number 
of organizational databases, together with external data 
repositories, typically including governmental Census 
databases and other repositories containing relevant 
market indicators. Data warehouses provide historical 
snapshots of the organization, its evolution and its data 
capturing processes. Practitioners agree that data 
warehousing is often a daunting task, due to a number of 
factors including the difficulty with ensuring user buy- 
in, the data quality issues associated with the extraction, 
transformation and transportation (ETT) of the data, 
poorly defined user requirements and unrealistic user 
expectations, amongst others. 



FIGURE 1 

THE LEARNING PROCESS 
(Adapted from De Villiers, 1996) 



The teaching process 




Backward loop followed by the learner 
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In an attempt to assist students to experience these 
difficulties first hand, students were given the task to 
create a data warehouse through the utilization of a 
Census data repository. The Census data repository 
concerned the 1991 and 1996 Censuses of the Greater 
Pretoria Metropolitan Council area located in the 
Gauteng Province of South Africa. Approximately 3 
million people, residing in formal or informal settlement 
within rural and urban areas, inhabit this area. 

The data from the two Censuses were placed in a data 
repository totalling approximately 214 Mbytes and 
consisting of 23 tables. One of the features of this data 
repository is the fact that it is spatially orientated and 
includes summarised information. That is, the data are 
used in conjunction with a geographic information 
system (GIS). The tables thus include a GIS Key 
attribute, which denotes one of 2606 regions on the map 
of the greater Pretoria area. 

Table 1 depicts a subset of the tables contained in 
repository. The figure shows that, for each of the 
different regions, demographic (summarized) 
information regarding the inhabitants was stored. For 
example, the age, citizenship, first language, race and 
gender of the inhabitants, amongst others, were 
collected. In addition, the Census data warehouse also 
contained information regarding the living conditions 
within a region. For example, the type of fuel used for 
lighting (gas, open fire or electricity), the type of 
telephone facilities and type of refuse disposal were 
recorded. Most of the tables included information 
regarding the four population groups (Black, Coloured, 
White and Asian-Indian) together with the Gender. For 
example, Table 2 shows a table containing information 
regarding the incomes of the various groups within a 
region. For example, the table shows that, in region 
7010734, there are thirty Black Females with income 
between ZAR5 000.00 and ZAR10 000.00. 



TABLE 1 

CENSUS DATABASE TABLES 

Region and Country of Birth by Gender by Population 
Group 

Region by Age by Gender and Population Group 
Region by Individual Income (Monthly) by Population 
Group and Gender 

Region by Water Supply by Population Group and 
Gender 

Region by Type of Dwelling by Population group of 
Head of Household and Gender 
Region by Number of Rooms by Population Group and 
Gender 

Region by Disability Summary by Population Group 
and Gender 

Religion by Population Group 

Students were subsequently given the task to construct 
a data warehouse from the Pretoria Census data, as 
discussed next 

User Requirement 

The users’ levels of involvement in the development of 
the data warehouse included initial and follow-up joint 
application development (JAD) sessions, a number of 
consultation sessions, evaluation and improvement of 
the initial design of the data warehouse, together with 
the evaluation of the final product. This evaluation was 
based on the usability and performance of the data 
warehouse in terms of the data mining efforts. 

The users indicated that the main aim of the data 
warehouse is to aid them to analyse the quality of life in 
the different regions of Pretoria. This information 
supports the identification of additional interrela- 
tionships, basic trends and thus the possible discovery of 
new knowledge. For example, the interrelationships 



TABLE 2 

REGIONAL TOTALS OF MONTHLY INCOME FOR EACH GENDER AND POPULATION GROUP 



GISKey 


Race 


Gender 


Criteria 


Total 


7010734 


Black 


Male 


R5000-R10000 


44 


7010734 


Black 


Female 


R5000-R 10000 


30 


7010734 


Coloured 


Male 


R5000-R10000 


10 


7010734 


Coloured 


Female 


R5000-R 10000 


8 













between level of education, electricity supply and level of crime are of importance. These results will 
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subsequently be linked to health issues and crime 
statistics. In particular, the Census data warehouse will 
be used to aid the data mining effort regarding 
tuberculosis (TB) diagnosis. In particular, the question 
whether there is a correlation between standards of 
living (as per this data set) and TB occurrence will be 
addressed. Students were therefore also given the 
(optional) choice to incorporate the South African 
Public Health Database into the data warehouse. 

The quality of human life depends on the quality of his 
environment. According to the South African 
Demographic and Health Survey of 1998 (Department 
of Health, 1998) there can be a correlation between the 
human, his environment and background characteristics, 
and the prevalence of disease. In the report only the 
prevalence of symptomatic asthma and chronic 
bronchitis are documented and it seemed important to 
the user to determine whether the same would be true 
for the occurrence of TB cases. 

Non-urban participants of the survey, report the 
symptoms of the above-mentioned diseases more 
frequently than their urban counterparts. South Africa is 
divided into nine provinces, namely Gauteng, Western 
Cape, Eastern Cape, Northern Cape, Free State, 
KwaZuluNatal, Mpumalanga, Northwest and Northern 
Province. For both diseases the Northern Province 
reported the lowest rates whereas North West Province 
reported the lowest rates for chronic bronchitis in 
woman. Women more frequently than men, and older 
people rather than younger persons, report symptoms of 
asthma and chronic bronchitis. Low levels of education 
are strongly associated with high rates of both 
conditions, but differences by ethnic group were 
reportedly small 

By using the Census data, which contains various 
background and environmental indicators, one should be 
able to determine certain contributing factors to low 
quality of living and link these to a certain region and 
probably to the occurrence of disease. 

The users thus indicated that the students were expected 
to design a data warehouse containing data that could be 
used to determine a region’s quality or standard of 
living. Comparing these with the TB prevalence data 



from the same region thus would enable healthcare 
workers and planners to identify problem areas within 
regions. 

After evaluation of the original data repository, 102 
areas within the Pretoria region were selected to be 
included in the data warehouse. These areas ranged from 
rural to urban, with a mixture of population groups and 
genders. Seven of the original tables were selected that 
had to be used by the students in the building of the 
warehouse, since these tables were thought to be 
indicators of quality of health. The criteria selected as 
indicators ranged from fuel used for cooking, to number 
of rooms per household, income, refuse disposal type, 
toilet facilities, water supply and household size. 
Students were required to use this information to 
construct a data warehouse concerning the quality oflife 
of the inhabitants, as discussed next. 

OVERVIEW OF PROJECT 

Students were required to complete the proj ect in groups 
of two or three, using the DBMS of their own choice. 
The project consisted of two stages, as follows. 

Stage 1: Initial Planning and Design 

The first stage involved the initial planning and design 
of the system. The aim of this first part was for students 
to determine the scope and boundaries of the system. 
The students were required to hand in a report 
describing the organizational strategy and the objectives 
of the project, provide definitions to define the scope 
and boundaries of the main aims of the project, analyse 
the organizational requirements and objectives and 
design the data warehouse. The design phase involves 
formalizing the business rules, the data warehouse 
dimensional model (star scheme), modelling of 
summation and time, as well as the creation of the initial 
physical model. 

Stage 2: Implementation and Data Loading 

The second stage concerned the actually implementation 
of the data warehouse using a relational DBMS. The 
majority of students used Oracle 8i, while others used 
SAS, SQL Server and MS Access in conjunction with 
Visual Basic, Delphi or Crystal Reports. Here, the 
students were required to build the warehouse through 
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the creation and testing of the database structure, 
extract, transform and transport (ETT) the data, and 
develop appropriate querying and reporting facilities. 

In order to successfully complete a data warehousing 
project, a number of processes need to be addressed 
throughout the development. It should therefore be 
assured that the business requirements are being met; 
sound data acquisition approaches are followed; the 
most appropriate data warehouse architecture is 
selected; the high quality of the data within the data 
warehouse is assured; data administration and data 
access control take place; metadata policies and 
procedures are in place; thorough documentation is kept; 
adequate testing and training are scheduled; transition 
policies are in place and post-implementation support is 
provided for. Students were thus also required to 
complete (and document) these processes throughout the 
data warehouse project’s two stages. 

The success of our approach was evaluated by 
considering the level of cognition achieved by the 
students, as discussed next. In addition, the data mining 
group assessed the quality of the data warehouse by 
determining the usability thereof, measured by the 
amount of information they were able to obtain from it. 

FEEDBACK/EVALUATION OF LEARNING 

It is the opinion of the authors that the simulation of a 
real world scenario provided students with an 
understanding of not only data warehouse concepts but 
also the difficulties surrounding data warehouse success. 
We believe that the success of the method lies in the fact 
that students are forced to reflect on their actions and 
consequently develop ‘professional artistry’. Although 
it is difficult to determine to which extent students 
attained these skills, students’ learning was evaluated on 
different levels, as discussed next. 

The student’s efforts were evaluated by determining 
how well the users’ expectations were being met. That 
is, the success of the exercise was measured by 
considering whether the business needs were satisfied. 
The majority of the projects did meet the need of the 
users, where the users where able to obtain relevant 
information from the systems. The level of user 
satisfaction ranged from 50% to 95%, with an average 
of 78%. 



The actual methodology used was also assessed, i.e., it 
was determined whether the students actually used a top 
down or bottom up iterative approach, or a combination 
thereof. This was determined through unstructured 
interviews during the project demonstrations, as 
discussed next. The majority of the students employed 
a combination of the top down and bottom up 
approaches. That is, the user requirements were used to 
develop the initial data warehouse conceptual design 
(star schema). However, students agreed that they also 
consulted the database itself when creating this schema. 
This practice corresponds closely to real-world data 
warehousing efforts, where the developers need to take 
both the user requirements and the data constraints into 
consideration during the design and implementation 
stages. 

Students were also given the opportunity to, through 
questionnaires, share their experiences of the data 
warehouse development process and voice their opinion 
on the factors necessary to ensure data warehouse 
success. The questionnaire is in attached in Appendix A. 
Tables 3 to 5 summarize the responses received from the 
sixty students who participated in this study. 

Table 3 shows that students were made aware of the 
importance of clear user requirements to successfully 
complete a data warehousing effort. Most students 
agreed that clear user requirements are paramount to 
ensure the success of a data warehousing effort. It is 
well known that the clear understanding of user 
requirements is difficult in information system analysis 
and design. This fact is augmented in a data 
warehousing project where users usually wish to obtain 
“ad hoc results through “what if analysis. 
Interestingly, the students found the user expectation to 
be frequently unrealistic. This also corresponds to the 
real world environment, where practitioners agree that 
the management of user expectations is difficult. 
According to Berson and Smith (1997), it is not 
uncommon for data warehousing system to fail to meet 
the high expectation of its users. This fact is mainly due 
to the domain specific nature and summarized nature of 
the information contained in most data warehouses. 

In the second part of the questionnaire, students were 
requested to arrange eight common mistakes, which 
influence data warehousing success in order of 
importance. The resultant ordering is depicted in Table 
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TABLE 3 

PERCEPTION OF USER REQUIREMENTS BY STUDENTS 





Always 


Frequently 


Seldom 


Never 


A verage 


Successful data warehousing depends on clear user 
requirement 


42 


14 


i 


i 


3.6 


Clear user requirements more important than user 
involvement 


3 


24 


23 


10 


2.3 


User expectations unrealistic 


12 


36 


2 


0 


2.7 



4. The table shows that the students did realize the 
importance of clear user requirements, the importance of 
ensuring high quality data and the difficulty associated 
with the ETT processes. The table also shows that the 
students generally underestimated the technical 
requirementsof the project, another frequently occurring 
phenomenon in real-world datawarehousing project. 

TABLE 4 

COMMON MISTAKES 

INFLUENCING DATA WAREHOUSE SUCCESS 

1. No user involvement throughout the development 
process 

2. Loading the data warehouse with irrelevant data 

3. Underestimating the technical requirements 

4. Starting without the right executive sponsor 

5. Choosing a technology-oriented rather than a user- 
oriented manager 

6. Believing data warehousing design is the same as 
transactional data warehouse design 

7. Delivering data with overlapping and confusing 
definitions 

8. Having unhappy users 

Students were also givoi the opportunity to provide 
anonymous feedback with respect to the usefulness of 
the data warehousing project. Table 5 shows some of the 
feedback received. The table shows that the students 
generally perceived this as a challenging, but 
worthwhile project. During informal discussions, many 
students agreed that the ETT processes were the most 
time consuming and tedious part of the data 
warehousing effort. Especially, the transformation of the 
data from its “horizontal spatial format to the “vertical 



fact and dimension tables of the star schema was time 
consuming. The rather vague user requirements were 
also perceived as a problem. Interestingly, the users 
found the quality of the majority of final projects 
outstanding. That is, the projects produced by the 
majority of our students surpassed the users’ 
expectations. Many students developed graphical user 
interfaces (GUIs) through which the users were able to 
formulate their own “ad hoc queries and this perform 
“what if analysis. The users were therefore satisfied 
that the majority of their queries regarding the quality of 
life in the region Pretoria could be answered and that 
their requirements were being met. 

TABLE 5 

SOME FEEDBACK RECEIVED 
FROM THE STUDENTS 

The data warehousing project proved to be an exciting 
learning experience and utterly unforgettable. 

It was a good learning experience, but better user 
requirements would have helped. 

First encounter with this type of project gives one a 
better understanding of the concepts behind a good 
data warehouse design. 

It was very interesting.... 

A good learning experience, just abit confused with the 
user requirements at first. 

The knowledge gained was significant and relevant. 

A challenge... very relevant experience. 

Good experience: it was tough! I had a real mind shift. . . 

Getting the data “right was much more effort than I 
anticipated. It was really rewarding. 

CONCLUSION 
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Many lectures agree that it is difficult to teach students 
the complexities associated with large-scale projects, 
such as data warehousing, in a simulated classroom 
environment. The aim of the data warehousing effort 
described here was to provide students with a hands-on 
experience of the intrinsic difficulties of this approach. 
That is, the approach was aimed atprovide students with 
the skills to make decisions under conditions of 
uncertainty, in order to prepare them for the uncertain 
zones of practice where user requirements are vague and 
the quality of the data repositories are not ideal. It is the 
opinion of the authors that the method described here 
aided students in achieving the highest level of cognition 
of Bloom’s taxonomy, by providing the student to be 
able to apply his knowledge in an experienced matter. In 
the words of one of the students, “the data warehousing 
project proved to be an exciting learning experience and 
utterly unforgettable. 
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APPENDIX A 
QUESTIONNAIRE 



Please answer the following questions by using the given scale: 

4 3 2 1 

Always/Definitely Frequently/ Nearly almost Occasionally/ Seldom Never 

1. The success of a data warehouse project depends on clear user requirements. 

4 3 2 1 

2. User involvement throughout the data warehouse development is not so important provided that the user 
requirements are clear. 

4 3 2 1 

3. Users usually have unrealistic expectations from data warehouse solutions. 

4 3 2 1 

List the following common mistakes influencing data warehouse success in order of importance. That is, mark 
the mistake with the biggest effect as 1, the second most important as 2 etc. Base this on your experience and 
common sense. 

□ Underestimation of technical requirements (capacity, performance,...). 

□ Starting without the right executive sponsor and a good driver. 

□ No user involvement throughout the development process. 

□ Loading the data warehouse with irrelevant data. 

□ Believing that data warehouse database design is the same as transactional database design. 

□ Choosing a data warehouse manager who is more technology-oriented than user-oriented. 

□ Having unhappy users. 

□ Delivering data with overlapping and confusing definitions. 



P.T.O 

Do you think the Census data justifies the construction of a data warehouse? Give reasons for your answer. 
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Do you have any general comments on this data warehouse project? 



Thank you for your participation! 
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ABSTRACT 

Specializations within Information Systems (IS) are becoming common, with one of the specialized areas being End- 
user Support (EUS), but no specific curriculum exists for EUS. This paper discusses the development of an EUS 
course that combines theory and student projects to simulate the duties found in user support. EUS specialists must 
be prepared to provide technical support, train end-users, and assist in solving business problems. The course uses 
group projects, hands-on tasks, and work with end-users to develop the skills needed to be a successful support 
person. Continued efforts will enhance this course and help develop new courses to meet the diverse needs of this 
new specialized area within IS. 



INTRODUCTION 

The introduction of the personal computer (PC) has 
changed how we do business. Required functions within 
organizations, and the way they are accomplished, will 
never be the same. Software application programs are 
more capable than their predecessors, and these 
packages allow users to develop and maintain their own 
applications. Proliferation and networking of PCs and 
user-friendly application software has created an area in 
IS referred to as end-user computing. 

Researchershave addressed many issues associated with 
end-user computing, such as end-user satisfaction [7], 
management of end-user computing [1, 5], and the 
“how-to necessary to support end-users [4, 6, 8]. Most 
of these studies and IS programs themselves are geared 
toward the traditional systems development pro- 
fessional individuals who will most likely enter the 
workforce as a programmer or analyst. Preparing 
information systems (IS) professionals to provide End- 
user Support (EUS) has traditionally received little 



attention [2, 3]. Efforts have been made to identify the 
skills necessary for EUS specialists, and such programs 
are being developed. Details concerning a course 
designed to introduce students to a possible career 
specializing in EUS follow. 

THE NEED FOR End-user COMPUTING 

End-user computing can be defined as the direct hands- 
on use and/or development of IS by system users. Many 
reasons exist for the advancement of end-user 
computing, and the need for people with support skills 
continues to increase. As IS and information technology 
(IT) continue to permeate nearly all business settings, 
IS/IT departments struggle to meet the needs of users. 
End-users have discovered that they can control a 
portion of their destiny and meet their own needs by 
developing and/or purchasing their own solution(s). 
Other reasons for the advancement of end-user 
computing include lower cost and greater accessibility 
of software and hardware, and an increasing computer 
literacy of the business workforce. This increase in end- 
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user computing has led to many management 
challenges, one of which is providing an adequate 
support function to meet the diverse needs of end-users. 

END-USER SUPPORT 

EUS is a broad term encompassing many areas, and the 
responsibilities of an EUS specialist are very different 
from those of traditional systems development IS 
professionals. End-users need training on software, 
hardware, and procedures, just as much as they need 
technical support. Supports specialists must provide 
software training, system consulting, technical support, 
software documentation, hardware consulting and 
support, and development support. Support specialists 
must also have a diverse set of technical skills, including 
(but not limited to) hardware, software applications, 
networks, database systems, Internet, and operating 
systems. In addition, concepts such as problem solving, 
communication, and interpersonal skills play a vital role 
in the success of a support specialist. 

TABLE 1 

TOP FIVE AREAS OF JOB GROWTH RATE 
1998 TO 2009 

Job Titles Projected Growth 

Computer Engineers 108% 

Computer Support Specialists 102% 

Systems Analysts 94% 

Database Adm inistrators 77% 

Desktop Publishing Specialists 73% 

As Table 1 highlights, the top five areas of job growth 
from 1998 to 2008 are computer related, and computer 
support specialists rank second with a 102% projected 
growth rate [10]. Based on the InfoWorld 2001 
Compensation Survey (http://www.infoworld.com/pd/ 
2001 iwcs.html), the following characteristics were noted 
for the computer support position. An average salary of 
$44,973 was reported, with 49% feeling that those in the 
support positions are fairly compensated. The average 
bonus was about $2,100, with 50% receiving a bonus. 
Salary raises averaged 19%, with over 76% getting 
raises. The average work experience in IS was 8 years, 
and the average time spent with the current company 
was 4.5 years. While about 27% were high school 
graduates, the remainder of those reporting had 
completed at least an associate degree. Only 47% held 
some type of technology certification. Those surveyed 



reported that the three most important motivation factors 
were compensation, recognition from management and 
users, and working with cutting-edge technologies. 
Most support personnel worked about 43 hours per 
week. The two most noted web sites for data were the 
Help Desk Institute (www.helpdeskinst.com) and the 
Association of Support Professionals (www.asp 
online.com). 

EUS COURSE 

Researchers continue to identify IS skills necessary for 
successful IS graduates, and professional associations 
develop and modify model curricula. The IS model 
curricula, and thus IS degree programs associated with 
them, are oriented to the application development area 
in computing. Traditional systems development 
curricula prepare students to enter the job market as 
programmers with the skills necessary to develop, 
manage, and maintain IS or IT. In most development 
programs, the attention devoted to skills required to 
support the end-user computing environment is minimal 
or non-existent. Some of the necessary skills are 
tangential, in that students are expected to know content 
but are not taught the material in a formal classroom 
setting. 

Lee, Trauth, and Farwell [9] suggest different career 
tracks will emerge within the IS field as the dynamics of 
business continue to alter the frame workof organiza-tions 
and information technology. Their results also indicate 
that current IS curricula are inappropriately preparing 
students to meet the needs of business, and they suggest 
new IS curricula be developed to meet the needs of the 
various career tracks within the IS field. None of the 
reviewed curricula has a course specifically related to the 
EUS area. The closest situation is in the Organizational 
Systems Research Association (ORSA) material with 
parts of three courses related to this content. The three 
courses are end-user technology solutions (OEIS-2), 
organizational and end-user information systems planning 
and design (OE1S-3), and designing and managing 
organizational training (OEIS-5). 

While no specific curriculum exists for EUS, courses 
dealing with associated topics can be found in various 
academic institutions plus within national Information 
Systems (IS) and ORSA curriculum studies. The ORSA 
curriculum (www.osra.org) is designed for organi- 
zational and end-user information systems and thus 
promotes research and application of information 
technology in the end-user environment. Their obj ecti ve 
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is to support work processes, improve employee 
performance, and enhance overall organizational 
effectiveness in direct support of end-user-oriented 
goals and strategies. 

The course described in this paper is a junior level 
elective that has been developed to meet the specific 
needs for supporting end-user computing in today’s 
dynamic business environment. The course focuses on 
the three functions of a Help Desk or Information Center 
(IC). 

1. providing technical support on hardware and 
software applications, 

2. training end-users in the use of hardware and 
software, and 

3. assisting the organization in solving business 
problems. 

The prerequisites associated with the course are the 
software course in MS Office, Visual BASIC 
programming, and the systems analysis content. These 
prerequisite courses provide a solid technical foundation 
from which to build. Most students have also taken a 
programming course in C++ and/or COBOL. During 
the course, students are expected to apply the knowledge 
and experiences gained in their previous courses. 

Course Objectives 

Students apply knowledge gained through work 
experience and previous coursework to develop the 
skills necessary to expand their knowledge base. The 
course objectives center on students being able to: 

1. identify and understand the basic elements of 
support expectations from internal and external 
sources; 

2. describe technological concepts and tools used to 
serve the organization’s business needs, including 
management issues; 

3. prepare and present training sessions for business 
application tools; 

4. research and analyze end-user needs and provide 
appropriate solutions to meet those needs; 



5. troubleshoot hardware andsoftware associated with 
the microcomputer, including learning the skills 
necessary to function as a PC support or help desk 
technician; 

6. research, access, and assess appropriate web 
resources dealing with various support tasks; and 

7. evaluate products in a strategic setting for various 
business environments. 

Course Material 

When the course was first taught, no textbooks 
adequately covered the broad nature of EUS. While 
textbooks were available for applications, networks, 
operating systems, databases, training, and all of the 
other components, no one had developed a textbook 
specifically targeting EUS until Fred Beisse’s book, A 
Guide to Computer User Support (Course Technology, 
1999). A second edition of this book is now available. 
In the same manner, the new second edition of the 
O’Connor and Regan book, End-User Information 
Systems (Prentice-Hall, 2001), provides a synthesis of 
implementing individual and work group technologies 
within the EUS area. 

To compensate for the initial lack of a textbook, the 
instructors utilized journal articles, web sites, videos, 
guest speakers, tours, outlines, and casesto cover course 
content. In addition, former students who are currently 
working in user support roles have returned to share 
their experiences and show how class and work 
environments relate. By using these instructional aids, 
the course maintains a level of currency not available 
using only textbooks. 

Course Activities 

To achieve the objectives of the class, several 
approaches are used. Learning activities include 
lectures, class discussions, hands-on experience, cases, 
student presentations, and outside application 
assignments. Each student participates in a group 
project that includes learning a business application 
software package and instructing class members in its 
use. This portion of the course requires installation and 
documentation of the selected software; preparation of 
a training plan, tutorial, and exercises for student 
completion as a set of task-based assignments; and 
conducting a training session for the software. The 
results of the project are presented by the group and 
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must address issues concerning: the business problem 
addressed; choice, cost, capabilities, and use of the 
solution; resources required for training, implementa- 
tion, and maintenance; and recommendations 
concerning the software chosen. 

Another focus area concerns the student’s interaction 
with end-users. Several options have been explored to 
satisfy this requirement. Classmates may recruit 
students to help in local business settings. Local public 
agencies and private firms often desire assistance in 
developing their own training presentations and can 
benefit from assistance. For example, a student 
developed a one-day training session on Microsoft 
PowerPoint for employees of a local branch of 
government. Those individuals who attended were 
preparing to train other employees in use of the tool. 
While this type of situation may result in financial 
remuneration, the experience gained from interacting 
with end-users is even more valuable. Students can 
provide a valuable service to the community while 
discovering how to effectively assist and train others in 
utilizing IS resources. 

Other ways to achieve interaction with end-users are 
available on campus. Students may assist students in 
computer laboratories or campus users by working in an 
IT support department. Based on the university’s need, 
the instructor develops a list of positions available. 
Students then select their preferred venue for help 
sessions. A few students are chosen to work at the 
university ’shelp desk. One group of students developed 
a five-page tutorial on using Microsoft Visio for use by 
other students, faculty, and professional staff. Other 
students may be given a business problem and asked to 
develop prototypes of screens for implementation in 
Visual Basic. While support personnel are not typically 
considered programmers, they must understand the basic 
concepts of programming, including structured or 
modular coding or object-oriented programming. 

To develop their understanding of user issues with 
hardware, software, and networks, students work 
through hands-on computer projects. Older/replacement 
machines are brought into a computer laboratory. These 
machines are not screened beforehand, and hardware 
components may need to be replaced. In one situation, 
a student team went through four machines before their 
device was operational for projects. While the focus is 
on hardware down to the bare box level, many software 
packages are examined to enhance the troubleshooting 
and maintenance aspects in a “closed box environment. 



Common operating systems (MS-DOS, Microsoft 
Windows variations, and Linux) are installed, and basic 
tasks are completed. Additional work is done for the 
CMOS, IRQ, and DMA settings. 

A laboratory manual is used for the hands-on aspect of 
the course. While the projects are attempted more in a 
“closed box mode, the course allows hardware 
examination via the “open the box mode. Among 
seven books dealing with laboratory exercises [see 
Appendix B], Clint Saxton’s Enhanced Manual for 
Managing and Maintaining Your Computer , 3d ed. 
(Course Technology, 2001) was considered best for the 
course environment. Additional references concerning 
hardware support can be found in Appendix A. 

CONCERNS AND FUTURE IMPLICATIONS 

The skills associated with EUS are changing. Some 
technical skills, such as MS-DOS command level 
expertise, are disappearing due to the ubiquitous 
graphical user interface. For instance, operatingsystems 
and software packages rely less on the skills associated 
with system-level configuration of devices. Plug-and- 
play features take the hands-on skills away from the 
technician. The “old command level skills need to be 
included in the academic environment; otherwise, the 
basic skills will be lost for newer support personnel. 
“Old software, hardware, and operating systems are 
still in place and must be maintained. Interfaces 
between “old and current systems are necessary. 
Without command level skills, these tasks cannot be 
accomplished. 

Due to early exposure to PCs and the Internet, computer 
literacy among entering college students should be 
improving. Our experience indicates that Internet 
surfing, web page design, and e-mail skills are fine. 
However, in a rolling survey of over 700 students over 
the last three years, we find that students did not possess 
the skills required for tasks such as formatting disks, 
creating and moving folders, copying filesfrom diskette 
to diskette, and saving screen captures to files. This 
same set of skills was similarly weak for 200 graduate 
students at various stages of coursework in both 
computing and business programs. EUS must provide 
the technical support, training, and assistance needed to 
allow these future organization members to effectively 
perform their jobs. 

Student feedback from this elective course is starting to 
re-shape the content. Comments concerning the hard- 



Proceedings of the I6 h Annual Conference of the International Academy of Information Management 



123 



ware portion of the material have been very positive. 
However, graduates report that hands-on repair tasks are 
used more for personal machines than on-the-job. Their 
job-related hardware tasks have focused on swapping 
drives (floppy and CDROM) plus installing larger hard 
drives with the second drive becoming a slave (normally 
a swapping issue for the PC used as a stand-alone 
device). The other technical materials, while 
informative, have been used very little on the job. 

While certification is not an expected result for students, 
the success rate has been very good for alumni. Of the 
120 students who completed the course in the last two 
years, approximately 35 students have started their 
computing career in the support field. Twenty of those 
have attempted some type of certification (e.g., 
Microsoft, Novell, or CompTIA). Ninety percent of 
those attempting certification have passed the 
certification test on their first attempt. 

SUMMARY 

As the IS/IT field continues to expand, academic 
programs cannot prepare students to be proficient in all 
areas. We as academic IS professionals must prepare 
the general professional with some specializations. 
Fortunately, the specialized programs needed are 
beginning to emerge in IS programs. EUS specialists 
must be prepared to provide technical support, train end- 
users, and assist in solving business problems. 

Courses in EUS are an innovative approach topreparing 
IS professionals to support end-users. Fifteen different 
but similar approaches to teaching EUS were examined 
as the foundation for a new, specialized course. Input 
from these approaches provided a framework for this 
course designed to prepare students who plan to enter 
the job market as EUS specialists. While this class 
offers a great beginning, much work needs to be 
completed. Continued efforts will enhance this course 
and help develop new courses to meet the diverse needs 
of this new specialized area within IS. 
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APPENDIX A 

GENERAL HARDWARE SUPPORT 

Most of the books listed include at least one CD-ROM 

with sample questions, video clips, software, and other 

references. The following list of materials seems to 

have had a positive influence on teaching the class. 

1 . Andrews, J. A Guide to Managing and Maintaining 
Your PC. Cambridge, MA: Course Technology, 
2000, 1165 pages, ISBN: 0-619-00038-4. 

2. Chase, K. The IRQ Book. New York: The 

McGraw-Hill Book Company, 1999. 324 pages, 
ISBN: 0-07-134698-8. 

3. Johnson, D. and Stauffer, T. Upgrading and 
Repairing Your PC: ANSWERS , Certified Tech 
Support. Berkeley, CA: Osbome/McGraw-Hill, 
1998, 496 pages, ISBN: 0-07-882463-X. 

4. Lee, A. Building Your Own PC: Buying and 

Assembling with Confidence. Grand Rapids, MI: 
Abacus Software, Inc., 1998, 1 17 pages, ISBN: 1- 
55755-320-3. 

5. Meyer, M. A+ Certification Guide. New York: 
McGraw-Hill Book Company, 2000, 966 pages, 
ISBN: 0-07-212266-8. 

6. Minasi, M. The Complete PC Upgrade and 
Maintenance Guide , 9th ed. Alameda, CA: 
SYBEX, 1998, 1607 pages, ISBN: 0-7821-2357-0. 

7. Mueller, S. Upgrading and Repairing PCs, 1 1th ed. 
Indianapolis, IN: Que Publishing, 1998, 168 pages, 
ISBN: 0-7897-903-7. 

8. Olsen, M. [project manager]. Sourcebook for the 
Help Desk, 2d ed. Redmond, WA: Microsoft Press, 
1997, 477 pages. 

9. Rosch, W. L. Hardware Bible, 5 th ed. Indianapolis, 

IN: Que Publishing, 1999, 1415 pages, ISBN: 

0-7897-1743-3. 



APPENDIX B 

HANDS-ON LABORATORY MANUALS 

1. Antonakos, J. and Adamson, T. Microcomputer 
Repair, 3d ed. Upper Saddle River, New Jersey: 
Prentice-Hall, 1 999, 693 pages, ISBN: 0-13-893454-1 . 

2. Beeson, D. Assembling and Repairing Personal 
Computers, 2 nd ed. Upper Saddle River, New 
Jersey: Prentice-Hall, 2000, 458 pages, ISBN: 
0-13-081949-2. 

3. Evans, D. A+ Complete Lab Manual. Alameda, 

CA: SYBEX, Inc., 1999, 232 pages, ISBN: 

0-7821-2591-3. 

4. Mansfield, R. and Petroutsos, E. The PC Upgrade 

and Maintenance Lab Manual, Alameda, CA: 
SYBEX, Inc., 2000, 279 pages, ISBN: 

0-7821 -2707-X. 

5. Regan, P. Troubleshooting the PC. Upper Saddle 
River, New Jersey: Prentice-Hall, 2000, 655 pages, 
ISBN: 0-13-095796-8. 

6. Saxton, C. Enhanced Manual for Managing and 
Maintaining Your Computer, 3d ed. Boston, MA: 
Course Technology, 2001, 344 pages, ISBN: 
0-619-03435-1. 

7. Schmidt, C. The Complete Computer Repair 
Textbook, 2 nd ed. El Granada, CA: Scott/Jones Inc, 
Publishers, 2000, 745 pages, ISBN: 1 -57676-033-2. 
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ABSTRACT 

According to Dixetai, [1998] human-computer interaction (HCI) involves the design, implementation and evaluation 
of interactive systems in the context of the user's task and work. This paper will focus on the HCI evaluation 
techniques used in the design and implementation of interactive systems with the emphasis on electronic commerce 
sites. Electronic commerce is according to Turban et al., [1999] the process of buying and selling or exchanging of 
products, services and information via computer networks, which includes the Internet. Books on electronic commerce 
seldom, if ever focus on the evaluation of web sites [Turban et al., 1999; Kalakota & Whins ton, 1996; Dahl & 
Lesnick, 1996]. The fundamental question that needs answering is whether we can apply the existing HCI evaluation 
techniques to evaluate electronic commerce sites or do we need more than that or totally different techniques. Existing 
evaluation techniques that will be discussed with a view of applying them to evaluate electronic commerce sites are 
cognitive walkthroughs, heuristic evaluation, review-based evaluation, model-based evaluation, observational 
techniques and usability testing. In order to answer the question the paper will apply some of these HCI evaluation 
techniques to a number of South African electronic commerce sites. Proposals for the extension of these techniques 
to make them more applicable for electronic commerce, as well as the possibility of new evaluation techniques, will 
be discussed. 



INTRODUCTION 

The importance of evaluation in HCI design is 
mentioned by several authors of HCI books [Dix et al., 
1998; Preece, 1994; Schneidermann, 1992]. Preece 
[1994:601] says that “ Without doing some form of 
evaluation it is impossible to know whether or not the 
design or system fulfils the needs of the users and how 
well it fits the physical, social and organizational context 
in which it will be used.” In the electronic commerce 
field, especially in business-to-consumer commerce, it is 
essential to evaluate the success of the business on the 
web. If one, however, consult books written on 
electronic commerce there is a lack of discussion or 
mostly no discussion at all on the evaluation of 
electronic commerce sites [Turban et al., 1999; 
Whiteley, 2000; Kalakota & Whinston, 1996; Dahl & 
Lesnick, 1996]. 

This paper discusses the use of HCI evaluation 
techniques in an electronic commerce postgraduate class 



as a tool to teach the students how to evaluate business- 
to-consumer sites. The application of the various 
students of these methods to different South African web 
sites will be described. All sites were developed by 
leading South African web development companies as 
listed in Intelligence, a monthly computer magazine 
published in South Africa. The problems that came 
forward using HCI techniques will be highlighted and 
possible solutions and further research will be 
mentioned. The first part of the paper sets the scene by 
describing what is understood by electronic commerce 
and HCI evaluation. 

DEFINING ELECTRONIC COMMERCE 

Turban et al., [1999:4] describes electronic commerce as 
“an emerging concept that describes the process of 
buying and selling or exchanging of products, services 
and information via computer networks including the 
Internet” Some people use the concept of electronic 
business and then refers to a broader definition of 
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electronic commerce, which includes servicing 
customers, collaborating with business partners and 
conducting electronic transactions within an 
organisation. Electronic commerce will be used in this 
paper in its broadest sense. We normally classify 
electronic commerce by the nature of the transactions 
that can be business-to-business, business-to-consumer 
and intra-business or organisational electronic 
commerce. The focus of the evaluations done for this 
study is on business-to-consumer electronic commerce, 
which involves retailing transactions with individual 
shoppers and was done on the final web site that was 
published by the particular company and not during the 
design phase. 

HCI EVALUATION TECHNIQUES 
Defining Evaluation 

Preece [1994:602] defines evaluation as follows: 

Evaluation is concerned with gathering data 
about the usability of a design or product by a 
specified group of users for a particular activity 
within a specified environment or work context. 

Why do we want to do evaluation? The answer to this 
question is normally to find out what the users want and 
what problems they experience. Without evaluation the 
system that reaches the consumer will be untried and 
only reflect the ideas of the design team with no 
relationship between design and use [Preece, 1994]. 

During evaluation it is important to consider the 
characteristics of the users of the system, the types of 
activities the users will do and the nature of the product 
(prototype or fully developed system) being evaluated. 
We normally distinguish between formative and 
summative evaluations. In this study a summative 
evaluation of a fully developed product was done. 

According to Rubin [1988], there are three general 
objectives which can be identified in the evaluation of 
any user interface, irrespective of the type of interface, 
the hardware or software, the stage in the design, the use 
or not of humans in the evaluation process, the method 
used and the type of data gathered. These objectives are: 

• The assessment of the capabilities of the design against 
the requirements of the users. 

• The assessment of the impacts of design decisions on 



the user and the user’s interactions with the system. 

• The diagnosis of problems with the design in an 
attempt to identify and clarify the scope of specific 
problems. 

Review of HCI Evaluation Techniques 

Several authors described different evaluation techniques 
[Preece, 1994; Dix et al., 1998, Schneiderman, 1992]. 
These include observing and monitoring usage, 
collecting user’s opinions and surveys, experiments and 
benchmarking, interpretive evaluation, predictive 
evaluation, usability laboratories, field studies, cognitive 
walkhrough, heuristic evaluation, review-based evalua- 
tion, model-based evaluation, empirical methods, et 
cetera. 

The following methods, based on the work of Dix et al., 
[1998] and Preece [1994] were discussed with the 
students in the Electronic Commerce class. 

Cognitive walkthrough. During the cognitive walk- 
through the evaluator step through the actions required 
by the user interface to check for possible usability 
problems. The walkthrough requires four things: a 
description of the system, a description of the task that 
the user must perform, a list of the actions needed to 
complete the task and a description of who the users are. 

Heuristic evaluation. This method was developed by 
Jakob Nielsen and Rolf Molich and concerns the use of 
a set of guidelines or general principles to critique a 
decision that has already been made. The list of 
heuristics includes: visibility of system status, match 
between system and real world, user control and 
freedom, consistency and standards, error prevention, 
recognition rather than recall, flexibility and efficiency 
of use, aesthetic and minimalist design, help functions, 
and help users recognise, diagnose and recover from 
errors. 

Review-based evaluation. In review-based evaluation, 
a study of literature for evidence to support or refute the 
different aspects of the design is done. According to Dix 
et al., [1998] “ the review should take into account 
similarities and differences between the experimental 
context and the design under consideration 



Proceedings of the 16 th Annual Conference of the International Academy for Information Management 



127 



Model-based evaluation. Certain cognitive and design 
models provide a means to evaluate, for example, the 
GOMS model, keystroke model (at a low level) and the 
design methodology - design rationale - which provides 
a framework in which design options can be evaluated. 

Empirical methods. Although the use of an experiment 
is a powerful tool for evaluating design, only the basic 
principles were discussed with the students due to the 
lack of statistical background of some of the students. In 
this method the evaluator chooses a hypothesis to test, 
using certain subjects and variables. 

Observational techniques. Think aloud, protocol 
analysis and post-task walkthroughs were discussed. 

Query techniques. Two main types of query techniques 
were mentioned, namely interviews and questionnaires 

i 

Usability testing. The usability principles from HCI, 
namely leamability, flexibility and robustness [Preece, 
1994, Dix et al., 1998] were discussed with the students. 
Considering usability from a web site perspective: 
According to Keith Instone (http://usableweb.com/ 
instone) you need to follow five steps and then watch 
people carefully as they try to use your site. The steps 
are: Know your purpose with the site, find ordinary 
users, watch and learn, collect the data and go back to 
the drawing board. 



During the lecture on evaluation techniques Jakob 
Nielsen's top ten mistakes in web design 
(http://www.useit.com) was also mentioned as well as 
the principles of good digital store design as described 
by Dahl & Lesnick [1996]. 

APPLICATIONS OF HCI TECHNIQUES TO A 
NUMBER OF SOUTH AFRICAN ELECTRONIC 
COMMERCE WEB SITES 

Students in the fourth year (honours) Informatics course 
on electronic commerce were given the assignment to 
evaluate different web sites where business-to-consumer 
transactions take place. They had to choose at least one 
of the evaluation methods discussed in class. In total, 
seven different sites were given to them, divided equally 
amongst the students. All of these sites were outsourced 
to South African web development companies and not 
developed in-house in the company. Only three of the 
sites evaluated will be described in this paper, namely 
the site of Fortes King Hotels, AA Auto finance and 
Cellular Shop. The students’ background on HCI 
evaluation techniques were information given to them in 
a two-hour lecture and study material handed out in 
class. They have had extensive exposure to interface 
design, both in the undergraduate studies and in the 
design of web sites during the electronic commerce 
course. 
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Fortes King Hotels 

(http://www.fortesking-hotels.co.za) 

Fortes King is a South African company that manages 1 1 
hotels in the Western Cape Province. On the site, the 
user can browse through a list of hotels, see what the 
hotels look like and make holiday or conference 
reservations. The main objective of the site is to 
advertise and sell their services. The target users are 



people who want to go on holiday or seek a conference 
location. 

Evaluation of this site was done using an evaluation 
score sheet as described in the conclusion, cognitive 
walkthroughs, heuristic evaluation, think-aloud 
observational techniques, usability testing and post-test 
interviews. The summarised impression of the 9 student 
evaluators is given in Table 1. 



TABLE 1 

FORTES-KING HOTELS 



Evaluator 


Methods used 


Impression 


i 


Evaluation score sheet 
Cognitive walkthroughs 
Think-aloud observation 
Post-test interviews 


An average site with no serious errors, but rather dull. No 
entertainment, specials or new features that would prompt the 
customer to return to the site. 


2 


Heuristic evaluation 
Usability testing 


Design is simple and clear and looks professional but rather clinical 
and boring. No entertainment and is purely an information provider. 
Inconsistent in some places and there is room for improvement. 


3 


Heuristic evaluation 


The site is easy to use, understand and navigate. Includes the 
correct amount of information. The use of images is done very 
effectively. 


4 


Review-based evaluation 


The site has a professional appearance, easy to navigate and use, 
includes all the necessary information. 


5 


Usability testing 


The site can do with a lot of improvement: It does not add any 
information that could not be included in a brochure or given over 
the phone. It does not give Fortes-King a competitive edge. 


6 


Cognitive walkthroughs 
Heuristic evaluation 


The web site provided enough information to satisfy customer 
needs at a high level, screens effectively displays the company 
profile, but more advertising and less boring screens could be used. 


7 


Review-based evaluation 


Site is properly designed, maintained and developed. 


8 


Heuristic evaluation 


Content and context offered are very effective in providing the 
necessary information and means to enable customers to interact. 
More effort can be made to entertain visitors and rewarding of loyal 
customers. A very good web site. 


9 


Heuristic evaluation 
Use of Netmechanic 


Content of the site is sufficient but site navigation is difficult, much 
more can be provided on the site to expand their business. 
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AA Auto Finance 



( http://www.aaaf.co.za ) 




The AA Auto finance web site is aimed at the 
prospective vehicle buyer and gives a customer the 
opportunity to apply for vehicle financing online. It 
also aims to inform potential customers of services 
offered and to offer the initial stage of those services 
online. Eleven students evaluated this site and all of 
them chose to do a heuristic evaluation of the site. 
Their impressions of the site are summarised in Table 
2. 

Cellular Shop 

( http://www.cellshop.co.za ) 

The Cellular Shop is the only independent specialist 
retailer authorised to offer all network products from 
the two official cellular service providers Vodacom 
and MTN. They specialise in providing unbiased 
advice about communication requirements and the 



selling of cellular phone, satellite phones, PDAs and 
cell phone accessories online with free delivery of you 
purchase to your doorstep. The site has both a normal 
and a flash version. 

Ten students evaluated this web site and their 
impressions are summarised in Table 3. 
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TABLE 2 

AA AUTO FINANCE 



Evaluator Impressions 

1 The site is relatively well-designed, but includes some of the top ten design mistakes [Jakob 
Nielsen]. At first glance the site seems cluttered. 

2 The site is a pleasing and informative web site with all the basic information that is needed. 
Some minor design errors were encountered. 

3 The site is aesthetically pleasing, but lack on the content side. 

4 The site is technologically intensive and has an easy to use interface. The testing shows very 
positive results with easy access to information. Downloading time may be a problem. 

5 The overall impression is very good, except for slow response times. Layout was done very 
well. 

6 Use of colours on the site is not working, eg white on yellow. A number of errors were picked 
up, eg links that are not working, orphan pages. Overall impression is good and the use of 
graphics effective. 

7 It is a good site in terms of simplicity, provision of feedback and short loading time. There are 
erroneous links, empty pages and misleading heading links. 

8 This student used a rating sheet for the different heuristics and came up with 62% for content 
and 73% for design. His overall impression was good although some errors were reported. 

9 This student also used a point system and awarded the site with a mark of 20 out of a possible 
40 marks. His final comment: “It takes more than a pretty face to create successful web 
business 

10 Too many mistakes and oversights make this a fairly average to poor web site, due to an 
overall lack of content and interactivity. 

1 1 Positive: no long scrolling pages and overly long download times. Negative: links are not 
working and use of colour. 
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TABLE 3 

CELLULAR SHOP 



Methods used 

Usability testing 
Heuristic evaluation 


Impression 

It has an overall good design with a few problems. Pages are easy to 
access, navigation is well-designed, no broken links or orphan pages. 
Users are encouraged to return with news and links pages. 


Heuristic evaluation 

Review-based 

evaluation 


Site is quite good but people will probably use it only to collect 
information before going to a traditional store. Added services like 
choosing your phone and contract on the web is necessary. 


Heuristic evaluation 


The site performs satisfactory with respect to consistency and aesthetic 
and minimalist design. Some information is outdated and there are some 
orphan pages. 


Heuristic evaluation 


No proper design was carried out before the implementation. The site is 
not impressive at all and there are lots of common mistakes that web 
designers make. 


Heuristic evaluation 
Usability testing 


Overall impression was that the site is aesthetically pleasing and 
functional. The content is relevant and useful, it is easy to learn and is 
entertaining and interesting. An excellent site. 


Usability testing 


The student used a scale to rate the site and came up with a average 
value of 4.2 on a scale of 1 (not acceptable) to 5 (excellent). 


Heuristic evaluation 


There are interesting links which make this more than just a site selling 
things. It is a good site which can be awarded if they clear up some 
minor mistakes. 


Cognitive walkthrough 
Heuristic evaluation 
Review-based 
evaluation 


The site is functional and easy to use. The student rated the site as 7 / 
10. 


Heuristic evaluation 
Usability testing 


The site reaches the goals of the company, which is providing a 
professional image. The site is successful, scoring high on basic design 
principles and provides a functional and effective digital storefront. 


Heuristic evaluation 


On the whole a good web site with lots of information, but there are 
minor problems that should be solved to improve the company's image. 
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CONCLUDING REMARKS 
ON THE EVALUATION 

Judging from the different responses from the students, 
who are not experienced evaluators but knowledgable in 
electronic commerce and have at the stage when they did 
this assignment already completed the development of 
their own electronic commerce site, the evaluation 
process was not entirely successful. Most students chose 
to use the heuristic evaluation, which was probably the 
easy way out. They did not, however, use the same set of 
guidelines. Many of the students used the top ten 
mistakes mentioned by Jakob Nielsen on his site 
http://www.useit.com as a set of guidelines. Others used 
the set of heuristics mentioned by Dix et al., [1998], 
while there were also students who used the design 
principles of a digital storefront described by Dahl & 
Lesnick [1996] and any guidelines they could get on the 
web describing good web design in general. The use of 
these different sets of guidelines probably resulted in the 
difference of opinion about the three sites evaluated. As 
could be expected, students who used more than one 
evaluation method came up with far better results and a 
more complete evaluation than those who used only one 
method. 



CONCLUSION 

Software exists on the web that allows you to test your 
web site. Most of those packages and guidelines are 
general and not developed specifically for electronic 
commerce business-to-consumer sites. One such package 
is on the web site garage site (http://websitegarage. 
netscape.com). National Chamberweb also allows you to 
form an idea of the quality of your site (http:// 
vvww.chamberweb.com/evaluate) and concentrate more 
on electronic commerce type of applications. This 
evaluation tool considers the following sections: 

• Findability, where the site is rated on whether the right 
actions are taken to build traffic. One such action will 
be have the URL on all printed material and promote 
the site in advertisements, events and conversations. 
The site must also be registered with the main search 
engines such as Yahoo, LookSmart, Snap! and others. 
External links to your site must exist. 

• Relevance and features, which evaluates relevant 
content, member directory, two-way communication 
(eg electronic mail) and special features such as 
calendars, chat rooms, et cetera. 



IMPLICATIONS FOR THE TEACHING 
OF EVALUATION TECHNIQUES FOR 
ELECTRONIC COMMERCE SITES 

From the results of the evaluation it was clear to the 
author and lecturer that a two hour lecture on evaluation 
methods was not good enough to get valuable results 
from the students. We still feel that it is a good way to 
teach evaluation of electronic commerce websites, but 
that more time need to be spent on the subject in class. 
The development of a specific set of usability metrics for 
electronic commerce sites is also recommended. It is 
important that we do teach the students to do evaluation 
of their sites as this is a neglected part of electronic 
commerce teaching. In the course that we are currently 
teaching we have changed the course to include more 
time spent on HCI evaluation techniques. The students 
are also instructed to evaluate their own websites that 
they are designing, during the design phase and after 
completion of the site. These evaluations must be done 
using at least two evaluation techniques and must 
involve users of the electronic commerce site and not 
only the students. As this course is still running to the 
end of November 2001, the author will be able to report 
at the conference on the results. 



• Usability, covers a number of criteria including fast 
loading, text tags on graphics, visible navigation 
choices, compatibility with common browsers and 
navigation aids. 

• The company's objectives with the site are also 
evaluated and can include an increase in revenue, 
decrease in expenses and an enhancement of the 
existing features of the company. 

The following are features that need evaluation, but they 

are still relevant to all types of web sites and not 

necessarily electronic commerce sites. 

• Broken Links — a number of software packages are 
available to check the existence of broken links and 
construction pages such as InfoLink Link Checker 1 .9 
(http://www.biggbyte.com/download/html). 

• Testing of contact information and reply time — what 
is the turnaround time for queries and the quality of 
the answers? 

• Updates and maintenance — last date updated should be 
given and information should be recent and relevant. 



Proceedings of the 16 th Annual Conference of the International Academy for Information Management 



133 




149 



• Competition’s sites — a look at the main competitors 
may give you an idea of the standard of your site. 

Srivihok [2000] explains in his article that the evaluation 
of electronic commerce is critical for the success of 
future systems. He developed a tool based on the task- 
technology fit model, which focuses on user evaluation 
of information systems. Srivihok proposes that more 
systematic techniques are needed to measure user 
evaluation of electronic commerce web sites. In his 
methodology he uses he used confirmatory methods to 
develop a hypothesised model to randomly collect data 
from electronic commerce users surfing the Internet. In 
the second step he uses exploratory methods to 
empirically evaluate the data and thirdly confirmatory 
methods will be used to test the hypothesised model 
against new data collected. This is, however, still 
research in progress and is not necessarily that different 
from the use of empirical methods in HCI evaluation. 

From traditional literature, conference proceedings and 
publications on the web, it seems as if the focus stays on 
what is good web site design and what constitutes a 
usable web site. The evaluation of electronic commerce 
web sites seems to be considered of less importance. In 
a paper by Tilson et al., where they evaluated four e- 
commerce sites, they discovered that there are a number 
of new issues, but that these issues can still be 
understood within the context of traditional HCI 
principles. More research needs to be done on how we 
can use the HCI evaluation techniques, enhance them to 
make them usable for the evaluation of electronic 
commerce web sites. At the moment it seems as if 
electronic commerce developers think that if they teach 
good web design, no evaluation is necessary! 
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USING WEB-POSTED SYLLABI 
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ABSTRACT 

A review of syllabi available on the World Wide Web has been conducted to illustrate topics of universal interest, 
modest interest, or very little interest among academics . Some topics, such as the use of Entity-Relationship modeling 
were almost universally covered in these courses . Emerging topics such as Data Visualization and Multimedia DBMS 
had almost no coverage in the syllabi studied. Web interfaces, a topic which would be expected to have wide interest, 
had only modest coverage. The full results of this study should be of interest to all higher education faculty teaching 
an introductory database class. Some recommendations are made based on emerging trends identified in the 
literature of this study. 



INTRODUCTION 

Survey studies of Information Systems specialty areas 
have been instructive in identifying developments and 
trends at the course level as well as the program level 
(Sendall, 1999, O’Hara, 2000). In order for IS faculty to 
best serve student andemployer constituencies, sharing 
information about topical coverage, pedagogy, and 
emerging issues is critical. Few other fields have such 
rapid turnoverin currentcontent and richness of sources 
of knowledge. 

This study focuses on the undergraduate database course. 
Coverage of Database Issues has been identified and 
confirmed as an important part of a comprehensive IS 
Curriculum (Couger, et al, 1995, Longenecker and 
Feinstein, 1 999). No single area in IS has such universal 
applicationacross organization types and organizational 
levels (McFadden, Hoffer, and Prescott, 1999, 
Rindegard, 1999). Emerging topics such as Object 
Databases (Sgarioto, 1999) and Data Mining (Whiting, 
2000) pressure database curricula in universities to keep 
pace with industry. Identifying commonalities and 
trends in undergraduate database courses will expand on 
these important foundational curriculum studies. 



Integrationof databasesand web applications is a subject 
of high industry interest as a key component of 
Electronic Commerce (Morrison and Morrison, 2000). 
Most modern database products such as Access and 
Oracle 8i haveweb developmentfeatures built in. Some 
database texts have a focus on w eb pages as the prim ary 
user interface (Riccardi, 2001). Whether academia is 
keeping up with this apparent mandate will be of interest 
in the current study. 

This survey is descriptive in purpose. A broad survey of 
syllabi (mostly from Spring 2001) will establish a 
general model of coverage across most institution types 
and locations. It will also serve as a cornerstone for 
future research of a norm ative nature that would include 
information surveyed from employers about what topics 
are important and useful for persons entering the job 
market to know. 

METHODOLOGY 

The methodology for the study is a content analysis of 
on-line syllabi for database classes in four-year 
institutions. An initial list of syllabi was located using 
a web search tool with the words “database” and 
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syllabus.” This search yielded over one hundred 
locations. Universitiesfrom the United Statesas well as 
other countries are included. An initial convenience 
sample of ten syllabi was analyzed to list the relevant 
content items that w ould be recorded in the larger study. 
Twenty-seven items were identified. 

The investigator followed promising links in order to 
find syllabi that w ere deem ed appro priate for the study. 
The study was limited to four-year institutions offering 
Bachelor’s degrees and above. Where schools of 
business were clearly separate from computing science 
programs, the database course from the business 
curriculum were used. In a few cases, a “CS” course 
syllabus was used due to the absence of Information 
Technology programs or courses within the Business 
School. Most syllabi are from undergraduate courses. 
Some (usually identified by a course number of 500 or 
above) were from Master’s in Business programs. In 
these cases, itwas clear that the course was intended for 
MBA students who had no prior course in database 
principles. 

Extensive searching of the list of links, and further 
searches to expand the list, yielded 5 0 online syllabi. 
The syllabi were printed to facilitate analysis. Each 
syllabus was carefully studied, with the presence or 
absence of each of the 27 content elements of interest 
being recorded for all sample points. The content 
element had to be explicitly stated or clearly implied in 
order to be counted as present. Results were kept in an 
Access database. The database also recorded whether 
the institution was public private, and the country the 
institution was located in. Totals and subtotals were 
computed using Access queries. 

RESULTS 

Final results of the content analysis of all 50 database 
syllabi were consistent in proportion of presence of 
contentelements with the pre-test of 10 syllabi. The pre- 
test summary is given in Table 1. 

These preliminary results suggested that many ‘current’ 
database topics were receiving little coverage in 
university curricula. It was determined that this was an 
adequate list of content elements, even though it was 
likely that some syllabi in the final sample might include 
elements not listed here. The ten institutions used to 
ascertain this list were not shown above, as they are part 
of the total sample shown in Table 2. 



The twenty seven content elements and their frequencies 
for the full sample are given in Table 3. 

DISCUSSION 

Some interesting findings are evident in Table 3. As 
might be expected, topics such as ERD’s and the 
Relational model receive universal coverage. The most 
popular platform for official sanction by a course (where 
this is a preferred package ) is MS A ccess, with O racle in 
second place. Object Oriented modeling, web 
integration, and SQL are covered about half the time. 
Surprisingly, several topics, which were strongly stated 
to be important in the literature review, receive little 
coverage. For example, only about a fourth of the 
courses studied include Data W arehousing. Many topics 
identified in a widely recognized IS Curriculum study 
(Couger et al., 1995) receive far below full coverage. 
Examples are: Object Oriented Modeling (52%), 

Multimedia (8%), and structured development 
methodologies (60%). 

Comparisons of public vs private institutions, and US vs 
non-US institutions yielded some interesting 
observations. Most topics were covered equally across 
these divisions. However, there were a few distinct 
differences. For example, 64% of public institutions 
covered distributed databases, while only 36% of the 
private ones did. Generally, public institutions covered 
more topics than private ones. Non-US institutions were 
far more interested in Client-Server technology and 
Middleware than their US counterparts (75% to32%). 
US Institutions were slightly more likely to cover 
security issues (66% to 50%) and Database 
Administration (71% to 50%) than non US ones. 

IMPLICATIONS AND LIMITATIONS 

The most significant of these findings was the low 
occurrence of coverageof important topics, as identified 
by industry and academic literature. Faculty teaching 
database classes should be interested in the overall 
results of this study, either as a validation of what they 
are already doing, or a mandate to expand topical 
coverage in their courses. 

A further benefit of this study is to stimulate further 
research into appropriate content of database curricula 
either by grounding in prior studies or co llecting fresh 
primary data from industry and academia. While the 
current study was mostly descriptive, further research 
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TABLE 1 

RESULTS OF PRELIMINARY ANALYSIS TO DETERMINE LIST OF CONTENT ELEMENTS 



Topic 


Institution Number 




1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


Intro to Database Topics 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Intro to Relational Model 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Non-Relational DB Architectures 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


No 


Yes 


No 


Conceptual Modeling (ERD) 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Normalization 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes ' 


Physical Design / Performance 


No 


Yes 


Yes 


No 


Yes 


No 


Yes 


Yes 


Yes 


No 


Relational Algebra / QBE 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


Object Oriented Modeling 


Yes 


Yes 


No 


No 


No 


No 


No 


Yes 


Yes 


No 


System Development Issues 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


No 


Yes 


Data Flow Diagrams 


Yes 


No 


No 


No 


No 


No 


No 


No 


No 


No 


Applied M S Access 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


No 


No 


No 


Applied ORACLE 


Yes 


No 


No 


No 


No 


No 


No 


Yes 


No 


No 


Transaction Processing 


Yes 


No 


Yes 


No 


No 


No 


No 


No 


Yes 


No 


Overview of Distributed. Databases 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


No 


Yes 


Yes 


No 


Concurrency Control 


Yes 


No 


Yes 


No 


No 


No 


No 


Yes 


Yes 


No 


Client-Server / Middleware 


No 


No 


Yes 


Yes 


No 


Yes 


No 


Yes 


No 


No 


SQL Overview 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Applied SQL 


Yes 


Yes 


No 


No 


No 


Yes 


Yes 


Yes 


Yes 


No 


Security Data Integrity 


Yes 


No 


Yes 


Yes 


No 


No 


No 


No 


Yes 


No 


Database Administration 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Data Warehousing 


No 


Yes 


Yes 


No 


No 


Yes 


No 


Yes 


No 


No 


Data Mining/Knowledge Discovery 


No 


No 


Yes 


Yes 


No 


Yes 


No 


No 


No 


No 


Integrating Data with Web Applications 


No 


No 


No 


Yes 


No 


No 


No 


No 


No 


No 


Data Visualization 


No 


No 


Yes 


Yes 


Yes 


No 


No 


No 


No 


No 


Text Based DBMS 


No 


No 


No 


Yes 


No 


No 


No 


No 


No 


No 


Interface Design Issues 


No 


No 


No 


No 


No 


Yes 


No 


No 


No 


No 


Multimedia DBMS 


No 


No 


No 


No 


No 


Yes 


No 


No 


No 


No 



TABLE 2 

FULL SAMPLE OF DATABASE SYLLABI, WITH PROPORTION OF COVERAGE 



Case 


Course ID 


Course Name 


Funding 


Country 


1 


DS 354 


Database Management 


Public 


USA 


2 


BA 561 


Database Management 


Public 


USA 


3 


MIS 420 


Database Management 


Public 


USA 


. 4 


LS458 


Database Management 


Private 


Canada 


5 


COMP 5790 


Database Management 


Private 


USA 


6 


LSC 740 


Database Management 


Public 


Sri Lanka 


7 


INFSCI 1022 


Database Management Systems 


Public 


USA 


8 


COMP 380 


Database Management Systems 


Private 


USA 


9 


CS 352 


Database Systems 


Private 


USA 


10 


CMS 460 


Database Design 


Private 


USA 


1 1 


CMPUT 291 


File and Database Management Systems 


Public 


Canada 


12 


CIS 550 


Database and Information Systems 


Public 


USA 


13 


61004 


Database Systems 


Public 


China 


14 


DISC 7374 


Advanced Database Management Systems 


Public 


USA 


15 


950-770 


Advanced Database Design and Management 


Public 


USA 
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153 
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Table 2 
(continued) 



Case 


Course ID 


Course Name 


Funding 


Country 


16 


ISMT226 


Database Design and Administration 


Public 


China 


17 


COMP 378 


Introduction to Database Management 


Public 


Canada 


18 


1ST 213 


Advanced Topics in Data Management 


Public 


USA 


19 


DBS-M 


Databases 


Public 


UK 


20 


IDS 410 


Business Computer Technology 


Public 


USA 


21 


ISDS 473 


Database Management 


Public 


USA 


22 


ACCT & MIS 837 


Management of Corporate Data Resources 


Public 


USA 


23 


FMDS 421 


Business Database Development 


Public 


USA 


24 


MIS 325 


Introduction to Data Management 


Public 


USA 


25 


B20.3335 


Advanced Database Management 


Public 


USA 


26 


IS 321 


Principles of Database Management 


Public 


USA 


27 


MBA 780S 


Database Management 


Private 


USA 


28 


FMIS 3421 


Database Management and Design 


Public 


USA 


29 


MIS 114 


Database Management Systems for Business 


Public 


USA 


30 


6K-282 


Applied Database Management Systems 


Public 


USA 


31 


ISM 4212 


Information for O perating Control an d Data M gt. 


Public 


USA 


32 


MIS 484 


Database Management Systems 


Public 


USA 


33 


MI 210 


Database Management Systems 


Public 


India 


34 


LIS 5782 


Database Management Systems 


Public 


USA 


35 


MGT 245 


Database Management for Information Systems 


Public 


USA 


36 


MSC 3936 


Database Management Systems 


Private 


USA 


37 


CIS 8410 


Database Management Systems 


Public 


USA 


38 


MSIS 4013 


Database Management 


Public 


USA 


39 


CMPT 354 


Database Systems and Structures 


Private 


Canada 


40 


01528-0130 


Database Management Systems w/CASE Tools 


Private 


USA 


41 


CPT 272 


Database Programming 


Public 


USA 


42 


COM 3315 


Principles ofDatabase Systems 


Public 


USA 


43 


CSIS 3310 


Database Systems 


Public 


USA 


44 


91.573 


Database Management 


Public 


USA 


45 


INFS 3050 


Database Design and Inquiry 


Public 


USA 


46 


COSC 4411 


Database Management Systems 


Public 


Canada 


47 


INSS 651 


Data Base Management Systems 


Public 


USA 


48 


CSM 204 


Data Management Concepts 


Private 


USA 


49 


CSE 2132 


Database Systems 


Private 


Australia 


50 


52-355 


Databases 


Public 


Scotland 



with more solid evidence of need could be more 
normative in nature. 

Some significant limitations of the study are the 
convenience sample and the use of posted syllabi as a 
surrogate for more official content listings. No doubt, 



some of the courses covered topics that did not appear on 
the syllabi. Still, presence of a content element on the 
course syllabus is a good surrogate for degree of 
perceived importance to the faculty member, and hence 
the emphasis it will probably be given in class. 
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TABLE 3 

CONTENT ELEMENT FREQUENCIES FOR FULL SAMPLE 



Topic 


Total 


% of 


% of 


% of 


% of 


% of 




Found 


Sample 


Private 


Public 


US 


Non-US 


Intro to Database Topics 


50 


100 


100 


100 


100 


100 


Intro to Relational Model 


49 


98 


100 


97 


97 


100 


Non-Relational DB Architectures 


26 


52 


55 


51 


50 


58 


Conceptual Modeling (ERD) 


50 


100 


100 


100 


100 


100 


Normalization 


43 


86 


82 


87 


84 


92 


Physical Design / Performance 


37 


74 


64 


77 


74 


75 


Relational Algebra / QBE 


38 


76 


73 


77 


74 


83 


Object Oriented Modeling 


26 


52 


45 


54 


55 


42 


System Development Issues 


30 


60 


55 


62 


63 


50 


Data Flow Diagrams 


3 


6 


0 


8 


8 


0 


Applied M S Access 


24 


48 


36 


51 


53 


33 


Applied ORACLE 


13 


26 


9 


31 


26 


25 


Transaction Processing 


19 


38 


9 


46 


34 


50 


Overview of Distributed. Databases 


29 


58 


36 


64 


58 


58 


Concurrency Control 


19 


38 


27 


41 


37 


42 


Client-Server / Middleware 


21 


42 


27 


46 


32 


75 


SQL Overview 


44 


88 


91 


87 


87 


92 


Applied SQL 


33 


66 


55 


69 


66 


67 


Security Data Integrity 


31 


62 


55 


64 


66 


50 


Database Administration 


33 


66 


73 


64 


71 


50 


Data Warehousing 


14 


28 


9 


33 


29 


25 


Data Mining / Knowledge Discovery 


8 


16 


9 


18 


13 


25 


Integrating Data with Web 


14 


28 


36 


26 


29 


25 


Data Visualization 


4 


8 


9 


8 


5 


17 


Text Based DBMS 


8 


16 


18 


15 


13 


25 


Interface Design Issues 


2 


4 


0 


3 


5 


0 


Multimedia DBMS 


4 


8 


0 


10 


5 


17 
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REQUIRING STUDENTS TO BRING 
COMPUTERS TO CAMPUS: ARE UNIVERSITIES 
ACHIEVING THEIR GOALS? 



Cindy Randall 

Georgia Southern University 



ABSTRACT 

According to data from the Campus Computing Project, an independent annual survey of colleges' information- 
technology plans, less than 10 percent of colleges and universities currently require students to have their own 
computers . (“ Growing Number of Colleges ...,” 2000) However, as universities fight to secure a competitive 

advantage for both themselves and their graduates in the 21 st century, more institutions will be joining the ranks of 
those currently placing a desktop or a laptop in the hands of each co-ed . The goal of this paper is to poll the current 
thoughts of universities that do have such a requirement in place, of universities that are considering adopting a 
policy requiring computers in the near future, and of universities that have chosen not to jump on this band wagon . 

This paper will explore the technology objectives of these universities and attempt to determine if these objectives 
are being met through universities ' current computer ownership requirements . 



INTRODUCTION 

As the computer screen replaces the notebook, a growing 
number of universities, both private and public, are 
requiringthat students either purch ase or lease their own 
computers, adding at leastS 1000 per year to the average 
cost of college expenses (Port, 1999). For freshman at 
the University of North Carolina at Chapel Hill, the 
choice was whether or not to pay $21 82 for a standard 
ThinkPad or $2924 for a model with expanded memory 
and a larger hard drive. On some campuses of the 
Polytech nic University in New York, students are paying 
$500 each semester to lease a laptop (“Growing Number 
of Colleges... 2000). 

Some schools are requiring the purchase of specific 
hardware and software. Others are supplying minimum 
computer configurations appropriate to different degree 
programs. (University of Florida: Student Computer 
Requirement’s web site, 2001). 

While the students may bear some of the financial 
burden, states are investing heavily to create a 
technology-enabledenvironment. In Massachusetts the 



Board of Higher Education approved a $123 million 
program that includes vouchers forlow- income students 
and training, facilities, equipment, and academic 
programs (“Massachusetts Takes Step..., 2000). The 
plan unconditionally embraces technology at a time 
when educational specialists are still trying to ascertain 
how computer ownership impacts education. The Board 
of Higher Education deemed this to be a necessary step 
in ensuring graduates will have the technological skills 
to fill tens of thousands of business and high-tech job 
openings (Healy, 2000). However, this program has 
stalledfor lack of funding and public support. The board 
had anticipatedfinancing the program with state-issued 
bonds. The state IT division ruled against bond 
financing. Because computers rapidlybecome obsolete, 
monies will have to come out of general revenues. 
Events such as the evaporation of the state’s surplus and 
the public’s reaction to the cost have stalled the Board of 
Higher Education’s plans (Olsen, 200 1). 

Benefits 

All students will have the same academic advantage. 
Typically students who qualify for financial aid are less 
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likely to have the means to purchase computers. If 
technology is a university requirement, the cost can be 
factored into financial aid packages. Some universities 
are using special endowments or have redirected some 
revenue monies to help students meet these costs 
(“Growing Number of Colleges...,” 2000). 

The network and communications that these computers 
provide can improve faculty/student communications. 
Skills acquired while using the computers would provide 
real skills which would make graduates more marketable 
(Graf, 2000). Software could be obtained at reduced 
prices with a campus site license (Gates, 1998). 

Courses offered in all majors will be able to incorporate 
the use of technology (University of Denver: 2001 
Laptop Specifications web site, 2001). On some 
campuses, freshman are required to attend training 
sessionsso thatthey will be able to use and configure the 
computers and will be prepared for class work and/or 
assignments (Lawrence, 1999). 

In the case of wireless LAN technology, students can 
access a course material or e-mail faculty while sitting 
on a bench outside the library. Many believe that the 
greatest benefit of requiring computers comes from 
outside the classroom, where students have 24-7 access 
to campus network and the Internet (Olsen, 2001) 
Technology has elbowed its way out of computer labs. 
Universitiescan reclaim scarce physical space thatonce 
housed these labs. No more spending of endless dollars 
to rewire old building with asbestos ceilings and cinder 
block walls to meet changing technology needs. At 
Wake Forest this year, freshman received a wireless 
Ethernet card with their PC notebook, and a bill with 
their tuition statement. Approximately 1 1 percent of 
Carnegie Mellon’s students purchased wireless LAN 
cards with which to access the university’s wireless 
network (Brewin and Cope, 2000). 

Faced with stagnant enrollment and low SAT scores, 
Western Carolina University implem ented a plan to wire 
campus and require computers in 1995. Students may 
choose to purchase a desktop or a laptop. While 
enrollment has not climbed, the quality of the student 
has. SAT scores are on the rise as WCU emphasizes 
communications skills and critical thinking, not 
computer hardware (Johnson, 2000). 

While most administrators agree that they are unable to 
measure the real impact of computer ownership on 



education, this policy will impact the institution’s 
bottom line. Spending on information technology can be 
greatly reduced for the institution as these costs are 
passed on to the student (Olsen, 2001). 

Concerns 

Not everyone agrees that student ownership of 
computers is necessary. Everyone does agree that 
networks must be improved to accommodate increased 
traffic, the student’s cost of educatio n will rise, and both 
faculty and students will need additional training (Graf, 
2000 ). 

Security and battery life require thoughtand planningfor 
universities that wish to embrace wireless technology 
(Brewin and Cope, 2000). Some faculty also worry 
about the integrity of using high-tech electronic devices 
in the classroom (Kobin, 2000). 

Others are concerned about the added financial burden 
to students (Port, 1999). ChrisDuckenfield, viceprovost 
for computing and information technology at Clemson 
University, believes that it is unfair to require students to 
spend money on technology that will be obsolete in two 
years. In 33 labs across campus, Clemson has 800 
computers available that can be configured to suit the 
user. Students do not have to invest in their own 
machine, they do not have to lug machines around, and 
equipment cannot be stolen (“Clemson Develops 
System,” 1998). 

Students at some universities that require computer 
ownership have been disappointed. The computer 
requirement brought with it the expectation that 
computers would be used in some, if not all, classes. In 
fact, few classes are computer intensive. Restructuring 
courses takes commitment and tim e for faculty mem bers 
to create course content that is electronic (Olsen, 2001). 

RESEARCH QUESTIONS 

One goal of this paper was to survey universities with 
computer requirements and determine what their 
objectives were when these universities decided to 
embrace a technology requirement. The survey 
instrument then asked if these objectives are being met, 
and if not, why not. The instrument also asked the 
universities if they were experiencing any unexpected 
outcomes, good or bad, and if so what these outcomes 
are (i.e., change in enrollment or retention, etc.). 
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Another goal of this paper was to poll universities who 
have not begun requiring students to arrive on campus 
with computers in tow. Are they consideringimposing a 
technology requirement? If so, what are the driving 
forces behind this decision? If not, why not? 

A driver behind this project was the strategic planning 
process currently underway at the authors’ university. 
The information technology plan task forcehas directed 
the university to consider requiring students to 
purchase/lease laptop computers. This requirement 
would supplement or replace the current environment 
which makes 1400 computers at various locations on 
campus available for student use almost continuously 
during academic sessions. In addition, most student 
residence halls are wired for connection to the campus 
LAN. Plans are underway to make older residence halls 
“connected” via wireless technology. 

METHODOLOGY 

Although the authors are primarily interested in the 
computer ownership requirements at peer institutions, 
we decided to use the Chronicle of Higher Education 
listing of colleges/universities as our frame (“How the 
Classification...”, 2000). A stratified random sample of 
20 percent of the frame was selected from doctoral- 
granting institutions (total of 151 extensive, 110 
intensive), master’s colleges and universities (total of 
496 at level 1, 1 1 5 at level II), and baccalaureate colleges 
(total of 228 liberal arts, 321 general, and 57 
baccalaureate/associate). The provost/academic vice 
president at each selected institution was contacted via 
e-mail with a link to a web-based survey. If electronic 
contact could not be made, a letter was sent with an 
enclosed hard copy version of the survey. A follow-up 
contact was made via e-mail/mail approximately two 
weeks after the initial contact. 

RESULTS 

Forty-one private institutions and 59 public institutions 
responded to the survey. These 1 00 responses out ofthe 
295 surveyed renders a response rate of 33.9 percent. Of 
these 100 colleges and universities, 23 are doctoral- 
granting institutions, 39 are master’s colleges and 
universities, and 38 are baccalaureate colleges. Most (62 
percent) were institutions of 5 000 students or le ss. Table 
1 showsthe breakdown of respondents by size of student 
body. 



TABLE 1 



Size of Student Body 


Frequency 


less than 5000 


62 


5000 - 10,000 


19 


10,001 - 19,999 


9 


20,000 - 30,000 


5 


over 30,000 


5 



Thirteen of the 100 institutions that completed the 
survey instrument required students to own or lease their 
own computers. Of those requ iring all students to have 
their own computers, 64 percent make this requirement 
for students in every degree program. The remainder of 
those surveyed have this requirement in place for only 
some of their programs. 

One-third allow students to lease computer equipment, 
40 percent require that students purchase computer 
equipment. The remainder of institutions which require 
students to have computers leave the decision to buy 
versus lease up to the students. Two-thirds of these 
schools specify that the computer must be a laptop 
computer. 

Roughly 62 percent of those mandating students must 
have their own computers have had this requirement in 
place for 2 to 3 years. Eighty-three percent believed this 
endeavor has been successful. None of the institutions 
considered the mandate to not be successful. 

Of those notcurrently requiring computer ownership by 
students, most were not considering changing thispolicy 
in the near future. For a breakdown of responses, see 
Table 2. 

Chi-square goodness of fit tests were conducted for those 
institutions which have computer requirements in place 
to allow a comparison by institution (private versus 
public), type (doctorate-granting institutions, master’s 
colleges and universities, baccalaureate colleges), and 
size of student body. Only one of these tests yielded a 
differencethat can be considered to be significant. Fifty- 
six percent of those schools with a student body 
between 10,001 and 19,999 require students to come to 
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campus with computers. All other categories of schools 
based on size of studentbody had fewer than 1 2 percent 
of responding institutions mandating that students must 
own or lease a computer (p = .002). 



Table 2 



Future Plans 


Percent 


Currently planning to 
implement computer 
requirement 


3.6 % 


Currently considering 
to implement computer 
requirement 


27.4 


Not considering such a 
requirement 


69.0 



CONCLUSIONS 

This study did not culminate in any definitive results. It 
did provide us with someinitial information concerning 
the percentageof institutions that aremand ating students 
arrive with computers and some insight into the 
successes, failures, concerns, and limitations of these 
programs. 

In our survey instrument we asked for a contact person 
at the university who would be willing to talk with us 
and provide a clearer picture of whatis happening at the 
institution. Our next step in this research will be to 
contact these people and discuss their decision whether 
or not to mandate that students have computers and the 
reason(s) their institution made this decision. For those 
who chose to require computers, we would like to know 
the goal of their program and whether or not it is being 
achieved. 
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APPENDIX 

TECHNOLOGY REQUIREMENT QUESTIONNAIRE 



DEMOGRAPHICS 

1. Your institution is: 
41 % Public 
59% Private 



2. Type of university/college: 

23% Doctorate-Granting Institution 
39% Master’s Colleges and Universities 
38% Baccalaureate Colleges 

3. Size of Student Body: 

62% Less than 5,000 
19% 5,000 - 10,000 

9% 10,001 - 19,999 
5% 20,000 - 30,000 
5% More than 30,000 

COMPUTER REQUIREMENT 

1. Does your institution require students to own/lease 
their own computers? 

12.9% Yes (Please answer questions 2 - 6.) 

87. 1% No (Please skip to question 7) 

2. Are all students required to have their own 
computers or only students in some specific 
programs? 

64.3% All 
35.7% Some 

3. Computer equipment for students is: 

33.3% Leased 

40.0% Purchased 
26.1% Either 

4. Students are required to own/lease the following: 
66.7% Laptop 

0,0% Desktop 
33.3% Either 



5. Your institution’s computer requirement has beenin 
place for: 

23.1 % Less than 1 year 
61.5% Two to three years 
1 5.4% Four to five years 
0.0% more than 5 years 

6. In your opinion, considering what your institution 
hoped to achieve by requiring students to have their 
own computers and the actual results being 
experienced, this endeavor has been: 

5 8. 3% V ery Successful 
25.0% Successful 

1 6.7% Neither successful nor unsuccessful 
0.0% Unsuccessful 
0.0% Very Unsuccessful 

Now please skip to question 8 

7. Although computers are not currently required at 
your institution, such a requirement is: 

3.6% Currently being planned 
27.4% Currently being considered 
69.0% Not being considered 

8. Would you be willing to share details/opinions of 
the compu ter requirem ent currently enforced atyour 
institution? 

□ Yes 

□ Via telephone 

Number: 

Best time of day: 

□ Via e-mail 

E-mail address: 

□ Not at this time 

THANK YOU FOR YOUR INPUT. IF YOU WOULD 
LIKE A COPY OF THE RESULTS, PLEA SE E-MAIL 
US AT crandall@gasou.edu. 
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ABSTRACT 

The increased social fragmentation in the fast-paced lives of individuals, particularly college students, is resulting 
in many dysfunctional effects — disappearance of community feeling, loss in the ability to relate to one another, 
impaired ability to collaboratewith others, and greater obstacles sharing and co-constructing knowledge. Technology 
is repeatedly offered as an answer. However, because existing technologies are designed by applying theories of 
economic efficiency and technological determinism, the implementation often results in further fragmentation. 
Oldenburg's idea of the “third place" provides a refreshingly new metaphor to conceptualize how technology- 
mediated interactions can enhance, not reduce, the sense of community. In the specific context of IS education, an 
on-linethird place can not only enhance the sense of community among students, but it can also prepare students for 
careers in learning organizations operating in complex, dynamic, and global environm ents. Our preliminary attempts 
to implement the third place within an IS program seem to indicate that students accessing an on-line thirdplace are 
able to experience relational developm ent, collaboration, and cohesion. Such experiences are no t only cons istent with 
the notion of a community, but are also helpful in developing human resources that are suitable for distributed 
learning organizations of the future. 



INTRODUCTION 

Consider the following situation. 

A student is having trouble understanding the 
difference between the concepts of partial and 
transitive dependencies covered byhis database 
professor. He looks through his textbook, re- 
reads his notes but still can’t grasp the 
difference. The exam on normalization is at7:45 
{ a.m. the next day, so in desperation he calls a 
friend in the class and asks for her 
understanding of the concepts. They agree to 
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meet at the local coffeehouse and discuss the 
concepts. At the coffeehouse, many other 
students from the class (as well as from other 
MIS classes) are also present. They naturally 
break up into small groups of 3 or 4 and begin a 
conversation on concepts related to database 
design. Leaving the coffeehouse a few hours 
later, the student reflects on this coffeehouse 
experience. His frustration and confusion have 
disappeared, and he feels a sense of 
accomplishment in talking through the different 
concepts related to this course, and tying in 
topics from other courses as well. He expects to 
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perform well in the exam the next day. What is 
more, he seems to have enjoyed the social 
interactions. The conversations in the 
coffeehouse have allowed him to blow off 
steam, relate to others with similar interests, 
whether academic, extracurricular or 
professional, and feel a sense of connectedness 
with his classmates, many of whom will be his 
friends, his professional peers, and “contacts 
through his entire life. 

Shouldn’t every MIS student have the opportunity to 
take advantage of such a coffeehouse? Wouldn’t the 
education process be significantly enriched if such an 
opportunity could be provided systematically and 
consistently as part of many MIS programs, especially in 
those that have been forced to become impersonal in 
trying to cope with the enormous enrollment growths 
without matching resources? Also, recognizing thetime 
pressures that a majority of students face through much 
of their college life, would it not be a worthwhile 
endeavor to provide students with access to such 
coffeehouse environm ents with minimal space and time 
constraints where they can be part of a community? 

In this paper, we describe the notion of “third place 
proposed by Oldenburg (1989) as a way for enabling a 
sense of community among students, and provide a 
preliminary discussion on our experiences in utilizing a 
web-based technology to provide a “third place for 
information systems (IS) students. 

The importance of creating a community of peers has 
never been more pronounced, especially for college 
students. Over 3 8% of college students work while 
attending college (Lipke, 2000). This leaves a student 
little time but to run from class to work to home during 
a day. In addition, the increasing level of technological 
mediation in almost every activity has left our lives 
seriously fragmented (Speier, Valacich, & Vessey, 
1999), devoid of time and opportunity for interacting 
with others (Kenway & Bullen, 2000). Such 
fragmentation appears to be having a number of 
dysfunctional effects. It promotes unsociable as well 
anti-social behaviors, raises barriers for students in 
acquiring certain kinds of knowledge, and inhibits their 
ability to attain mutual understanding with others, a 
fundamental requirement in any cooperative activity, 
especially systems design and implementation 
(Churchman & Schainblatt, 1 96 5). Thus, the availability 
of a forum for informal interaction is necessary not only 



for the interest of attaining knowledge but also for 
ensuring a stable and just society. 

Focusing specifically on IS education, we argue that the 
development of community has numerous outcomes that 
are notnecessarily availab le in more traditionalmethods 
of deliveringIS courses. Most trad itional IS courses help 
students increase their understanding of certain technical 
knowledge, and acquire technical skills focused on 
developing, managing and maintaining operating 
systems and applying certain programming languages. 
While certain types of knowledge (propositional 
knowledge) can indeed be acquired through traditional 
classroom lectures, other types of IS knowledge 
(conceptual and symbolic knowledge) cannot be 
satisfactorily learned without interactions within a 
community setting (Sarker, Lau, & Sahay, 2000). We 
believe that creation of an online thirdplace such as one 
resembling a coffeehouse described in the opening 
scenario can be an effective strategy to facilitate student 
interactions, and thus, learning. 

The remainder of this paper is organized into three 
sections. First, we describe the third place concept and 
the role of technology in developing an on-line third 
place. Next, we describe our experiences in developing 
an online third place as part of an undergraduate 
database systems course at a medium-sized state 
university in the inland northwest region of the US. 
Finally, we discuss the implications that the results of 
this case study have on the future of IS education and on 
the development of IS courses. 

THE “THIRD PLACE” CONCEPT 

Study groups are not new in IS education, but the 
creation of a unique environment called a third place is 
new (Schuler, 1996); providing students with the 
opportun ity to converse on a wide variety of topics, and 
to debate and dissentin an on-line space. Ray Oldenburg 
(Oldenburg, 1989) describes a third place as a space 
where community members gather and interact. In his 
description, a firstplace is home, a second place is work, 
and a third place is considered a “hang out space. For 
example, a physical third place would be a coffee shop, 
a bar, a beauty parlor, or a drugstore soda counter. 
Oldenburg suggests that people seek out a third place 
because the functions it serves are missing from their 
lives. Building a community, with common bonds and 
ties, social interaction, and shared location (Hillery, 
1955) in an educational environment provides students 
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with a similar “hang out space and also teaches them 
how to create an informal public life themselves. 

Conversation is the main activity in a third place and 
occurs with a spirit of inclusion rather than exclusivity 
(Hamman, 1997; Schuler, 1996). In addition, 
conversations in the third place enable debating, 
dissenting, and keeping democracy alive (Atkin, 1998). 
These dialogs provide an opportunity to flush out 
questions, sound out protests, and form opinions in 
other words, develop a shared though critical 
understanding of issues. Members of these third places 
know they can gatherand be assured that acquaintances 
will be present at almost any time of day or night. These 
spaces welcome everyone and no one is expected to play 
the role ofhost or guest. Thisneutral ground encourages 
people to gather, not to talk to the owner or manager of 
the shop, but to converse with other patrons. Therefore, 
third places, as conceptualized by Oldenburg, are more 
than mere physical spaces. They require the interaction 
and companionship of people. The key to the existence 
of a third place is the neutral ground they provide for 
conversation that is not based on social or economic 
status and is away from home and work (Oldenburg, 
1 989; Rheingold, 1993). 

i 

For many of us, and this is especially true for college 
students, the opportunity to interactin a third place such 
as a neighborhood coffeehouse is limited. Our sense of 
community has declined as we have become more 
suburbanized. With this decline in community comes a 
lack of social cohesion and satisfaction in our society. 
Oldenburg suggests that a lack of community is 
poisoning not only an individual’s wellbeing but also 
grass-roots democracy and civil society. Certainly, these 
are important concerns for the higher education system 
that is charged with producing not only individuals with 
necessary skills but also individuals who can play the 
role of responsible citizens within their communities. 
The rapid expansion of Internet technology provides an 
excellent opportunity to address the difficulties 
associated with time-space fragmentation by extending 
Oldenburg’s third place concept to an online platform 
(Garton, Haythornthwaite,& Wellman, 1997). Vannevar 
Bush and Douglas Engelbart’s vision of the original 
computer is closely aligned with an online third place. 
The computer was seen as a tool to expand our 
intellectual powers. Every new step in technology, 
whether a new medium or changes within a medium, has 
brought different ways of distributing the message while 
still maintain ing and preserving aspects of the old system 
(Postmes, Spears, & Lea, 2000). The Internet enables the 
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creation of a com munity environm ent for learn ing that is 
relatively seamless for the learner and reflective of the 
work environment they will experience in the future. 

LEARNING AND THE(ON-LINE) THIRDPLACE 

A third place promotes originality and provides 
stimulationto the members. Online communication adds 
several unique components to a third place: a diverse 
population, a novelty of conversation that is predictable 
yet unpredictable, the opportunity to converse in an 
unselfish and a selfish manner simultaneously, and the 
dialectical experience for an individual of being a 
participantas well as a spectatorat the same time (Atkin, 
1998). Although most online interactions are primarily 
conversational and text based environments, they can 
promote a third place feeling (Harasim, 1990). In fact, 
many scholars have persuasively argued that such 
conversational and text-based environments greatly 
enhance learning. Papert (1996) contends that we learn 
by doing, and we learn even more if we combine our 
doing with talking and thinking about what we have 
done and what could be done. Similarly, Vygotsky 
argues that the process of articulating thoughts into 
written speech involves deliberate analytical action. He 
calls this change from “inner speech or verbalization to 
written speech a deliberate structuring of the web of 
meaning. Gage adds (Harasim, 1990) that writing is 
thinking made tangible. It is a way of holding a thought 
still enough to examine its structure and its flaw. 
Learners in an online environment report increased 
attentiveness to the content of a written message over a 
verbal message. Text offers an equalizingeffectallowing 
everyone to respond regardless of social, emotional, or 
physical considerations (Harasim, 1990). 

Instead of being a distant observer in a large lecture 
room, students in such environments are likely to be 
actively involved in the material (Smith & MacGregor, 
1992). Higher academic achievement and advanced 
cognitive development have been shown to occur with 
social interaction as opposed to individualistic learning 
activities (Bosworth & Hamilton, 1994; Johnson & 
Johnson, 1994). Settings where students are free to move 
around and discuss their thoughts and ideas with their 
peers enhance the learning. There are many ways or 
methods of using collaborative learning but all the 
strategies are centered around the students’ process of 
investigation and their discovery or application of the 
knowledge, not a teacher’s presentation of the material 
(Bosworth & Hamilton, 1994). Working in a group, each 
member is expected to be a responsible participant. An 



Proceedings of the 16 h Annual Conference of the International Academy for Information Management 



ERIC 



164 



on-line learning environment provides the tool to allow 
multiple intellectual minds to join together (Bruffee, 
1995; Bosworth & Hamilton, 1994; Cyganowski, 1990; 
Smith & MacGregor, 1992). 



designed to “simulate a coffeehouse-like environment, 
and provide the feel of a third place. The environment 
provides both an asynchronous and synchronous option 
for flexible interaction. 



A virtual third place also spawns a learning community. 
Interaction and communication are essential to the 
development of an effective learning situation. Learning 
communities foster student engagement and 
achievement; a core setting where everyone has equal 
access and an informal network (Gabelnick, MacGregor, 
Matthews, & Smith, 1990). Online third places attach 
value on diversity within the group and foster internal 
communication to promote caring, trust, and teamwork 
(i.e., relational skills) in addition to sharing task related 
skills, all critical for organizations to be successful. 
Students in a learning community interact with the 
content material, the instructor, and other students. 
Working cooperatively with other students allows them 
to look at something from a different perspective, and 
understand their role as an individual within the group 
(Kowch & Schwier, 1997; Lenning & Ebbers, 1999; 
Meisel & Marx, 1999). Conversation, dialogue, and 
debate provide a perfect union of egoism and altruism. 
Although activity in a third place is largely unplanned 
and unscheduled, a learnin g comm unity structure can be 
loosely structured around topics of interest. Senge’s 
“learning organizations (Senge, 1990), which are 
believed to be models of organizations that can excel in 
today’s complex, dynamic, and globally competitive 
business environments, have much in common with 
Olderburg’s idea of third place within the educational 
setting. Senge’s vision was for employees to expand 
their capacity to create results that they truly desire, 
nurture creative thinking, and help people learn how to 
learn together. We believe that an on-line third place can 
be a fertile training ground for future knowledge workers 
(including, but not limited to, IS professionals) who are 
likely to find themselves in learning organizations. In 
addition, we feel that the sense of belonging to a 
community of peers can, in many respects, minimize 
anti-social acts such as malicious hacking which has 
been on the rise in recent times. 

AN IMPLEMENTATION OF THE THIRD 

PLACE AND PRELIMINARY ANALYSIS 

An experimental online learning environment called 
Speakeasy Studio & Cafe was used for to create a third 
place for students in an undergraduate IS course. 
Metaphorically developed on a cafe setting, the space is 



Participants in our case study were undergraduate IS 
students enrolled in a database design course that was 
being taught by the first author. Most of the students 
participating in our study were in their last semester of 
study before graduation. The interaction among the 
students within the on-line space provided by Speakeasy 
was the primary focus of our investigation. It is worth 
noting that the online environment (the third place) was 
provided as a supplement to the face-to-face interaction 
during class times (the second place), and was not 
intended to be a substitute. 



Students in the course were given access to the system. 
To get the specific cafe assigned to the course, a student 
would log on to the Speakeasy system and then select the 
specific “neighborhood representing a broad sub-group 
(e.g., the relevant department within their college) with in 
the university community. Within the neighborhood are 
various cafes or courses at the university. Upon entering 
the cafe assigned to their course, students would see 
different scheduled events, some events currently 
playing, and others scheduled for the weeks thatfollow. 



FIGURE 1 

SPEAKEASY STUDIO & CAFE 
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Each event was assigned a table that represented an 
activity, which related to the event or topic being 
covered. Tables, complete with coffee cup rings for 
visual appeal, become the space to discuss and debate 
issues related to the course. A table can also be a work- 
space for a team to use for a project (see Figures 1 & 2 
for screenshots of the interface). The dialog between 
students at each table served as th e data for ou r efforts to 
identify preliminary themes related to the creation and 
outcomes of a third place environment. Results of our 
analysis are reported in the following section of this 
paper. 
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FIGURE 2 

SPEAKEASY STUDIO & CAFE INTERACTION SPACE 
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Next we discuss some preliminary themes that emerged 
from our analysis of student conversations within the 
“third place created using an online environment, 
Speakeasy Studio and Cafe. 

Analysis of On-line Conversations 

From the on-line conversations, we derived themes 
pertinent to the third-place concept by drawing on 
methodological guidelines from the grounded theory 
(Glaser & Strauss, 1967) and ethnographic traditions 
(Van Maanen, 1988; Agar, 1986). We discuss three 
inter-related themes that emerged from the data, which 
we see as central to the idea of the third place. The 
themes from this analysis are labeled: relational 
development, collaboration, and cohesiveness (see 
Figure 3). Student comments related to each theme can 
be found in Table 1. The themes identified here are 
consistent with the ideas of a shared locality, social 
interaction, and common bonds or ties that are seen to 
facilitate community development (Hillery, 1955). 

Relational development. Working in groups/teams is 
an essential part of our lives, more so now with the 
added opportunity ofdistributed te ams in the workplace 
(Lipnack & Stamps, 1999). Several aspects of our work 



include group interaction. Although just gathering 
people together in one space provides no guarantee that 
this collection of individuals will become a community, 
with the development of a relationship among these 
individuals, their shared experiences do tend to coalesce, 
thereby facilitating the formation of a shared sense of 
community. The first theme identified was relational 
develo pme nt relate d to gro up form ation. 



FIGURE 3 

THIRD PLACE THEMES 
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TABLE 1 

THEMES AND STUDENT COMMENTS 



Themes 


Examples 


Relational 

Development 


Student 1: What I would give to become a ski bum. 

Student 2: I have a friend who worked at some resort in Vail. He waited tables at night and skied all day. 

He served people like Warren Miller and Jack Nicholson. 

Student: I like working in a group, especially if it is a good group, i.e. everyone does his or her share of 

the workload. 

Student 1 : Interesting story, good talking to you. 

Student 2: I was able to go to Japan, Delaware, and Virginia with the AF. I was stationed in DE for 3 

years... pretty much sucked. 


Collaboration 


Student: / think I still need to learn the value of working on a team. Trusting team members to not 

only do their share of the work but to do it well, rather than trying to do it all myself I need 
to realize the value of other people s input and perspectives and not focus so much on my 
own answers as the right answer. 

Student: I am a hard worker and I work well in groups. I have been subjected to many different people 

and personalities because of my background. I am eager to do well, and lam willing to work 
with anyone who wants to do well and is not afraid to put the time in to do so. 

Student 1: My POM internship was honestly a good experience and I really do recommend it. I feel I 

was at a disadvantage being a sophomore intern. It was a pilot program to see just how 
young can PO M majors be picked for internships. 

Student 2: You may not want to hear this, but thanks to those of you that interned in the pilot group 

[company] has made significant changes in the internship program. Of course it is not 
exactly the same in all of the FABs but similar. 

Student 1 : Actually I am glad to hear about the internship being even better. At least what we did helped 

them plan a better internship that you get more out of. I would only be upset if it hadn ’t 
gotten better each year ! 

Student: I hope that is what you guys were thinking. I had a question about the top two entities on 

your diagram. I believe (sp) it is a one-to-one relationship ...I assumed that the attribute was 
functioning as a foreign key, but then I realized that the primary key from the mandatory side 
needs to go to the optional side... If that ’s what you guys were thinking, just let me know and 
I will change the relation. 

Student: We meant to use [attribute l] throughout the diagram. We intended to use [attribute I] as a 

foreign key in the en tity Status. 

Student: What is the primary key going to be for the PLATE _APP entity there are no candidate keys 

listed. Maybe enter a permit_ID entity? 

Student: ** We would suggest turning the Student_D into a subtype with overlap. 

Student: If you have any comments or questions regarding this transform, please let us know. 


Cohesion 


Student: Thanks for the response. I look forward to kicking some serious butt in this class with you 

and Sam. I think we'll do great! 

Student: The team will not share information about our ideas or 

policies with other groups. 

Student: The team will not share information about our ideas or 

policies with other groups. 

Student: Wow, It is always great to find a fellow Linux User. I have used linuxfor both a desktop 

environment as well as a server. 



Group work provides students with a multitude of 
differentperspecti ves that they may not have considered 
before. For a group to be effective they must address two 
dimensions, task and relation (Rothwell, 1998). Forming 
a group provides diverse approaches to the task 
dimension because of what each individual brings with 



them (Dennis, Aronson, Flenin ger, Sc Walker II, 1999). 
The relational dimension can be linked to social 
interaction. Socialization of undergraduates is very 
important to the college experience (Newman, 2000). 
John Dewey provides a description of the importance of 
social interaction with the quote “Not only is all social 
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life identical with communication, but all communi- 
cation is educative (Burbules & Bruce, (in press)). The 
more students get to know each other, the more 
information they share about their non-academic life as 
well as their academic lives. Significant learning can 
occur in this situation, with students’ recursively co- 
constructing knowledge (Stiles, 2000). 

Community and education are dynamic processes where 
knowledge is not a product to be accumulated but an 
active process in which the learner attempts to make 
sense ofthe world. A learningcommunity fosters student 
engagement and achievement. Students who are actively 
engaged in the learning process have two things in 
common, shared knowledge and shared knowing 
(Lenning & Ebbers, 1999). Research suggests that 
students who have a passive, un-engaging learning 
experience do not develop depth of knowledge (Stiles, 
2000; Bruffee, 1993; Johnson & Johnson, 1994). 

Atthe beginning ofthe semester students in thedatabase 
design course were asked to post a letter ofintroduction 
in the online environment that included some mention of 
their team-oriented and project-oriented strengths and 
weakness (i.e. information pertaining to task and 
relational dimens ions). Students were to respond to their 
classmates’ introductionsand communicate online in an 
effort to form teams for the course project. A review of 
the initial conversations that took place in this online 
environment showed that conversation tended to be 
relationship-oriented rather than task-oriented. Consider 
for example the following conversation between two 
students about skiing. 

Student 1 : What 1 would give to become a ski bum. 

Student 2: I have a friend who worked at some resort 
in Vail. He waited tables at night and skied 
all day. He served people like Warren 
Miller and Jack Nicholson. 

This discussion had nothing to do with the strengths and 
weakness ofthe student’s ability on ateam. On the other 
hand, it provided insight into what the student enjoys 
doing outside of academic life, which could be an 
important issue in the development of a relationship, 
even professional. Couch (1996) refers to such 
(seemingly irrelevant) conversations as the exchange of 
evocative symbols and contends that collaborative 
relationships are enabled and strengthened through use. 
Providing a space for students to discuss about issues 
related to their personal lives is an important function of 
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a third place. Such social interactions are justifiable not 
only because they help in the formation/maintenance of 
productive professional relationships, but also because 
the college experience of students cannot be separated 
from the social interactions that are needed for the 
existence of society th at is not completely dysfun ctional. 

Yet, relationships in a professional setting can be 
meaningless without at least a background concern 
regarding the task dim ension as well. Transcripts of the 
third place revealed interspersed use of what Couch 
(1996) refers to as “referential symbols (conversation 
segments related to tasks, time coordination, etc.) with 
evocative symbols among students. An example of 
referential symbolism everP at an early stage of 
relationship development is: 

Student: 1 like working in a group, especially if it is 

a good group, i.e. everyone does his or her 
share of the workload. 

It is apparent in this message of self-presentation, that 
this student is signaling future (or potential) teammates 
his general comfort in working with others, though he 
also indicates his awareness of the importance of 
distributing the workloadand staying on task (Rothwell, 
1998), a fundamental concern in systems project 
management. 

Informality. A sub-theme associated with relational 
development is the infomiality of the writing. This sub- 
theme is present in the tone and style of the writingthat 
students use with each other. Casual language, in 
electronic text, is similar to “chit-chat you might have 
sitting at a table. This is also a way of enhancing the 
relational dimension. Informal conversation, just as in 
formal conversations, uses established norms of the 
community. In a formal setting you would introduce 
yourself and provide information about who you are. 
The students in this case were not strangers; many of 
them had had courses together in the past and some had 
even worked together on teams. Although the students 
had been aware of each other, they didn’t necessarily 
know each other well. They did use introductory 
greetings and descriptions about themselves (Barker & 
Kemp, 1990). Many of the follow-up questions were to 
garner more information about something that was 
shared. It might be new information such as an 
internship the student had experienced or inform ation on 
an awaiting event such as a birth in the family. Slang or 
the use of established abbreviations also displays the 
informality of the message. 
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Student 1 : Interesting story, good talking to you. 

Student 2: I was able to go to Japan, Delaware, and 
Virginia with the AF. I was stationed in DE 
for 3 years... pretty much sucked. 

Using casual language and slang (“sucked ) indicates a 
degree of comfort in the interaction. As Oldenburg 
suggests, people are seeking outinformal, public places 
such as a third place because it is missing from their 
lives. The tone and style of language used by the 
students in the interaction is supportive of that 
suggestion (Berge & Collins, 1995). 

Collaboration. A group needs to have both the task and 
relation dimensions balanced in order to be effective. A 
strong commitment to the project is also essential to 
effectiveness (Kirkman, Jones, & Shapiro, 2000). In 
order to be effective on the task and relational aspects, 
the group-members must be proficient collaborators. 
Collaboration is the next theme that emerged in this 
study. Collaboration is defined as a naturally occurring 
social act wherein peers not only work togethertowards 
a particular goal but also learn from one-another 
(Gabelnick et al., 1990). As the previous theme has 
addressed, interaction and the sharing of information is 
a way to collaborate with others. Collaboration is an 
active process, not one that takes place in isolation. 
Talking about information allows us to organize our 
thoughts and to make our ideas clearer. In a discussion, 
different perspectives are articulated and debated. The 
purpose is not to come to consensus to but to create 
knowledge and meaning individually (Bruffee, 1984; 
Bruffee, 1993; Johnson & Johnson, 1994). Collaboration 
in the form of group work increases a student’s 
engagement in the learning process (Johnson & Johnson, 
1994). 

One of the activities in this case was for students to 
collaborate on forming project teams. The students 
sought information about potential team members. In 
discussions with other students, a studentwould look for 
information to make a decision on group membership. 
This created an inclusive understanding of each other’s 
strengths and weaknesses. In the process of finding out 
information, the students asked questions and were able 
to identify with one another based on responses 
provided. 

A group that collaborates more and interacts on a 
personal level increases the relational dimension of the 
group as well. As a personal relationship develops, there 



is greater personal accountability (Johnson & Johnson, 
1994). A sense of belonging to a group also tends to 
boost the desire to learn. One student clearly articulated 
the dichotomy of teamwork. 

Student: I think I still need to learn the value of 

working on a team. Trusting team members 
to not only do their share of the work but to 
do it well, rather than trying to do it all 
myself I need to realize the value of other 
people's input and perspectives and not 
focus so much on my own answers as the 
right answer. 

Collaborationbecomes part of the balancingthat is done 
between task and relational aspects of a group. 
Balancing is more fragile in a technology-mediated 
environment where students are juggling classes, work, 
family, and personality preferences, and this may be 
transparentto other members in the group. Some groups 
that focus on the task and lose sight of the relational 
aspect are less effective than those that have shared 
purpose and collaborate on all dimensions (Rothwell, 
1998). 

Student: I am a hard worker and I work well in 

groups. I have been subjected to many 
different people and personalities because 
of my background. I am eager to do well, 
and I am willing to work with anyone who 
wants to do well and is not afraid to put the 
time in to do so. 

As the project proceeded, it was quite apparentthat peers 
were learning from otherpeers through debates, critical 
examinations of artifacts (ER diagrams), and 
constructive suggestions, as the following exchange 
shows. 



Student 1 : / hope that is what you guys were thmking. 

I had a question about the top two entities 
on your diagram. I believe (sp) it is a one- 
to-one relationship ...I assumed that the 
attribute was functioning as a foreign key, 
but then I realized that the primary key from 
the mandatory side needs to go to the 
optional side... If that 's what you guy s were 
thinking, just let me know and I will change 
the relation. 

153 



Proceedings of the 16 h Annual Conference of the International Academy for Information Management 



o 

ERLC 



169 



best copy available 



Student 2: We meant to use [attribute 1] throughout 

the diagram . We intended to use [attribute 
1] as a foreign key in the entity Status. 

It was also interesting to see that collaboration occurred 
not only within groups but across groups as well, 
thereby indicating formation of a broader learning 
community. However, we did notice that task-related 
conversations among members of different groups 
tended to be more formal and polite than among 
members of the same group. 

Motivation and support: Another aspect of collaboration 
is providing motivation and support for classmates. This 
became a sub-theme related to the collaboration. From 
the messages pro vided there is a strong sense of support 
for other students. Even with the informality and 
sarcasm in some of the written messages, there is a sense 
that students care about others in the class. The message 
could feature a question of what type of internship a 
student may have, or express amazement regarding the 
fact that a student can be employed, have a family, and 
be a student all at the sametime. This support seemed to 
providemotivation for students to continue, and perhaps 
to seek excellence. It also becomes a driving force for 
students to want to come back to an online space and 
further create a third place atmosphere. 

Student 1: My POM internship was honestly a good 
experience and I really do recommend it. I 
feel I was at a disadvantage being a 
sophomore intern. It was a pilot program to 
see just how young can POM majors be 
picked for internships . 

Student 2: You may not want to hear this, but thanks to 

those of you that interned in the pilot 
group ... [The company] has made 
significant changes in the internship 
program. Of course it is not exactly the 
same in all of the FABs but similar. 

Student 1; Actually I am glad to hear about the 
internship being even better. At least what 
we did helped them plan a better internship 
that you get more out of I would only be 
upset if it hadn 't gotten better each year! 

Collaborationbetween all classmates was strong during 
the team formation stage, but as the teams formed, there 
seemed to be much greater intra-group collaboration. 
One task that required students to work together across 



team boundaries was when each team needed to 
comment on the Entity-Relationship Diagram (ERD) of 
another team it was paired with, and transform this ERD 
to relations. The purpose of the activity was to push the 
students along the conceptual design continuum from the 
diagram to the transformation process. By working with 
another team’s diagrams, hidden assumptions made by 
the original formulators of the diagram could be 
unearthed. Interestingly, the between-group 
collaboration was supportive and not critical, as we had 
expected. Polite suggestions and questions regarding the 
decisions made were more typical of the between-group 
interaction. 

Student: What is the primary key going to be for the 

PLATE_AP P entity there are no candidate 
keys listed. Maybe enter a permit _ID entity? 

Student: ** W e would suggest turning the Student _D 

into a su btype with overlap. 

Student: if you have any comments or questions 

regarding this transform, please let us 
know. 

Members of the groups collaborated and created 
knowledge based on the understanding of the material. 
Each group varied on a continuum of learning, but most 
appearedeffectivein producingan adequate design. The 
across group environment may be viewed as simulating 
a learning organization to some extent, where all 
members of the organization contribute to and benefit 
from the shared knowledge, not just the ones within a 
particular team (as is characteristic of traditional 
hierarchical organizations with minimal knowledge 
creating and sharing ability). 

Cohesiveness. The final theme identified from the 
conversations of the students was cohesiveness. 
Cohesiveness is defined as the common bonds or ties 
that unify the group. This concept is key to the 
development of a community and third place (Anderson 
& Kanuka, 1997; Hay thornth waite, Kazmer, Robins, & 
Shoemaker, 2000; Oldenburg, 1989). Cohesion in the 
group is represented by the harmony among the 
members. In a group environment, the ability to work 
toward a goal often requires the cohesion among the 
members (Postmes, 2000). This is especially true when 
tasks are complex and interdependent, and require 
mutuality and trust among members to accomplish 
together. Mere coordination among group-members 
through the use of referential symbols is not sufficient 
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for collaboration in these cases; instead true 
collaboration can occur only when group cohesion 
exists, i.e., individuals in the group feel a sense of 
closeness. 

In our case, we could identify several instances ofgroup 
cohesion among students. 

Student: Thanks for the response. I look forward to 

kicking some serious butt in this class with 
you and Sam. I think we 7/ do great! 

The students in this case were well acquainted with each 
other from other courses, and had shared a common set 
of experiences, and this created a personal connection 
(Haythornthwaite, Kazmer, Robins, & Shoemaker, 
2000 ). 

Communication is the key to discovering common 
bonds. During one conversation, two students discovered 
that they shared a common interest in Linux. Without the 
opportunity to participate in a dialogue, students would 
not have discovered this common interest, and thus, 
would not find it easy to develop the level of cohesion 
that was achieved in this case. 

Student: Wow, It is always great to find a fellow 

Linux User. I have used linux for both a 
desktop environment as well as a server. 
However, lately I have been more intrigued 
by its server capabilities than anything else. 

This small thread of commonality led to a lengthy 
discussion of the uses and importance of Linux. 

It is important to point out that cohesion in a learning- 
community does not necessarily imply a consensus on all 
topics. Rather, the individuals knit cohesively in a 
learning community feel comfortable in surrendering 
their autonomy and their personal convictions for a 
favorable outcome of the group. Sharing leadership in 
the group createsgreater commitment to the decisionsof 
the group, and this further enhances individual students' 
communication and interaction skills (Meisel & Marx, 
1999). 

Interestingly, in many cases, cohesion was evident 
within but not across teams. For example, the charter of 
one of the teams explicitly stated that members must not 
share project-related information with other groups. 



Charter: ... The team will not share information 

about our ideas or policies with other 
groups... 

While this observation was initially interpreted by us as 
an indication that an on-line community had not been 
successfully formed, we gradually came to the 
realization that cohesion often relates to the micro level 
of our community. Competition at a micro-level seems 
to often override an overall cohesiveness for the macro 
level (Aquino & Reed II, 1998). Another important point 
emerged from our analysis of data related to cohesion. 
While technology does create the opportunity for us to 
become more global, it is clear that cohesion doesn’t 
extend to all levels. The interests of students seem to 
have coalesced with members of their immediate project 
group, and not so much with the members outside of 
their own group or micro community. We believe that 
this finding is notat all inconsistent with the notion of a 
community for example, we may have individuals in 
families, families in a neighborhood, and so on. While 
individuals in a neighborhood may indeed develop 
relations around certain issues, and even collaborate on 
certain issues, the degree to which there is a merger of 
interests within families and across families is usually 
not quite the same. Yet, individuals from different 
families are able to live together as a social whole. 

CONCLUSION 

Imagine a local cafe having a number of available tables 
and chairs. On any given day, many students leave their 
classrooms, work, or homes to visit the cafe. On some 
days, the group collects around a single large table 
discussing an up coming exam. On other days, this group 
of students might divide into smaller groups of three or 
four, each sub-group sitting at its own table discussing a 
different topic. A person listening in might observe one 
team discussing an upcoming project milestone. At 
another table, thediscussionmightbe focusing on social 
plans for the weekend. The dialogue that takes place 
among these team-members can stop and start, and go on 
for days or weeks as the individuals gather more 
information, think aboutthe issues raised or take time for 
other responsibilities in their lives. There is no coffee 
served at this cafe, students are welcome to bring their 
own; and there are no requirements that shirts or shoes 
must be worn. The manager of the cafe is always 
present, arranging thetables or planning the discussions, 
but rarely does the manager sit at the tables or get 
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involved in the discussions. The emphasis is on the 
students, their tasks, and their relationships. 

Whether this cafe is a physical entity or a metaphor 
created in an online environment, the space is one that 
offers a location that has the potential to serve as a third 
place. In this paper we have described the third place 
concept for developing community among IS students, 
and shown one application of web-based technology to 
create a third place. 

Based on our (first-time) experiences in enabling a third 
place, we would like to state the following tentative 
conclusions. 

The on-line third place can help students experience 
social interactions with peers without significant time- 
space constraints. For example, a student may find 
peers in the third place at any hour of the day without 
having to leave his/her home. 

This interaction can lead to a number of useful 
outcomes: It may provide students an opportunity to 
blow off steam, “hang out and relax by engaging in 
“chit chat. It may also help students develop the skills 
to identify with others in addition to developing a 
mutual understanding with others. These skills are a 
prerequisite for the health of a society, as well as a 
requirement for any kind of collaboratio n to occur, be 
it in the personal or professional arena. Finally, 
numerous conversations can be going on 
simultaneously as if forming a collage, with many 
individuals sharing similar interests with some of the 
otherparticipants in the conversations. Th e possibility 
of orchestrating a complex pattern of interactions in a 
face-to-faceenvironment is difficult if not impossible. 

Most importantly, interactions in the on-line third 
place can allow a sharing of knowledge among 
students and enable critical discussions on individual 
students’ work, while retaining the casual and non- 
threatening atmosphere. While we have not obtained 
any objective measure of enhanced learning this time, 
we dare say that students have developed a deeper 
understanding of the material by engaging in 
conversations on the material. 

We found that an on-line third place does enable 
relational development and collaboration; however, 
the entire community does not function as one 
cohesive unit, but rather exists as a loosely coupled 
conglomeration of cohesive sub-communities. One 



conclusion for this micro-community effect is that 
students did not have the opportunity to use an online 
space as a supplement until their final semester of 
college. Earlier introduction of a third place may have 
enhanced the level of cohesiveness in the class as a 
whole, and reduced the inherent (sometimes petty) 
competitiveness of some of the students. Increasing an 
emphasis on communication and conversation helpsto 
balance competitiveness and collaboration (Aquino & 
Reed II, 1998). A possible direction for future research 
is to implement a third place environment early in the 
students’ educational process, and to follow the 
changes in their attitudes and behaviors over time till 
they graduate . 

To conclude, we wholeheartedly agree with Oldenburg’s 
assertion that a third place, especially in this age of 
increased social fragmentation, can serve important 
functions for people that are missing from their lives. 
Building an online third place for education of IS 
students teaches them how to create an informal public 
life, and how to acquire knowledge actively though co- 
construction, not just passively by acting as a receptacle 
of knowledge encoded in a lecture delivered in a 
traditional classroom. The skills to adapt, develop 
relations, and collaborate with peers that are gained 
through participation in an on-line “third place are 
critical to the success of an employee in the dynamic 
work environment of today and tomorrow. There is 
value to being sociable, in learning from peers, and in 
developinga sense of community, and the sooner we can 
get students to recognize this proposition, the more 
meaningful their college education will be for them. 
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ABSTRACT 

Universities teaching information technology need to know what beliefs their students hold about the information 
technology job market and what happens when their students enter the work force. Surveys were sent to students 
and recent graduates in information technology majors to compare the beliefs of the former to the experiences of the 
latter regarding entry-level jobs in the information technology field. 

Despite anecdotal evidence that students * expectations are unrealistic, this study found that students ’ expectations 
and graduates * experiences generally matched. Both groups recognized the importance of general knowledge and 
skills, and ranked problem solving as the most important skill. They felt the university is doing a good job teaching 
them most of the important knowledge and skills. 



INTRODUCTION 

While the number of available jobs in Information 
Technology has declined somewhat in the past year, IT 
continues to be one of the most attractive areas for 
graduating students. In recent years there have been 
anecdotal stories of students starting at six figure 
salaries, making much more than their professors. Some 
students also believe that they can start in or quickly 
obtain positions as analysts and project leaders, rather 
than spend time in more typical entry-level jobs such as 
programmers and help desk support. As a result, 
concerns have been expressed that student expectations 
tend to be unrealistic. 

From the perspective of educators in the IT area, one of 
the major concerns is keeping programs up-to-date. 
Technology is changing rapidly, and while the basic 
concepts of programming and data management have 
not changed that much over the past 20 years, the tools 
used certainly have. There is a continuing need to keep 
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programs current. 

This study attempted to address these issues by 
surveying the students and recent graduates of IT 
programs at a mid-sized western Canadian university. 
This university offers two separate four-year degrees 
that prepare students for jobs in the information 
technology (IT) sector, aB.Sc. in Computer Science and 
a B.Mgt. majoring in Management Information Systems 
(MIS). Both degrees are typical of those offered at 
similar universities. The Computer Science program 
requires much more programming and emphasizes more 
technical issues. The MIS major studies both technical 
and behavioral issues in IT and includes courses in all 
areas of business, including accounting, finance, 
organizational behavior and marketing. 

This paper attempts to answer the following questions. 

What are the job expectations of current information 

technology (IT) students? 
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What skills and knowledge do IT students think are 

important in the current job market? 

What skills and knowledge have recent graduates 

found to be most important? 

What kinds of jobs are IT graduates doing? 

LITERATURE REVIEW 

Employment in the information technology sector has 
been growing rapidly. All businesses today feel the 
need to leverage technology in order to be competitive. 
Despite the recent downturn, there are apparently still 
600,000 unfilled IT positions still available in the U.S. 
(Joachim 2001). This demand has put pressure on 
educational institutions to expand their programs, but 
qualified professors are in equally short supply. 
Moreover, the constant changes in technology make it 
difficult for slower-moving universities to update their 
existing programs to dovetail with the market Some 
studies, such as Jacobson and Armstrong (2000), have 
attempted to study employer needs in this market. 

Equally interesting is the effect the market is having on 
student expectations. All IT students are aware of the 
skills gap so frequently mentioned in the media. Most 
cities now have independent schools offering “quick 
fix solutions that promise IT careers to anyone after a 
six month to one-year program. Students’ expectations 
are affected by these ads and the mention of 
technologies that are often not in place at universities. 

Despite this continual change, some general knowledge 
and basic IT knowledge requirements remain relatively 
constant. General knowledge and skills that remain 
pertinent through the turbulence of change are 
interpersonal communications; interpersonal behavior; 
oral, written, and multimedia communications; and 
management of IS projects (Doke & Williams, 1999). 
Becker, Gibson, and McGuire (1994) identify key 
theoretical concepts that should be included in 
information systems curriculum: systems analysis, 

systems design, software engineering, testing, database 
management, data communication, and decision support 
systems. Also, the balance between training and 
education must be considered (Vermond, 1999). IT 
training has a short shelf life; skills that are current 
today may be out-of-date tomorrow. Education has a 
longer shelf life because it teaches how to use tools to 
solve problems. Universities tend to focus on education 



over training. Current training will allow recent 
graduates to function in entry-level jobs, while 
education will help them later in intermediate-level j obs. 

One study found academic and industry respondents had 
different perceptions of market needs (Heiat, Heiat, & 
Spicer, 1995). The industry stated the most urgently 
needed skills were problem solving, PC skills, 
multimedia, networking, and Geographic Information 
Systems (GIS), while the academics felt the most 
needed skills were computer-aided software engineering 
(CASE) tools, Windows, Unix, and systems analysis. 
However, they did not attempt to measure the 
perceptions of students. 

Students intending to major in IT may have some 
misconceptions about the field (Mawhinney, Cale, & 
Callaghan, 1990). Students in this study thought they 
would be working alone on programming tasks when 
they graduated However, they expressed a preference 
to work with other people. The authors concluded that 
students were getting the mistaken message that IT 
careers were all in programming because of the focus on 
programming courses in their degrees. They suggested 
developing “creative approaches to the placement and 
content of programming activities in both the major and 
the career. 

METHODOLOGY 

The first step in designing the survey was to determine 
which types of knowledge and skills shouldbe included. 
To do this, 35 recent IT job postings for work term 
positions were analyzed. Students at the university have 
the option of completing a work term as part of their 
studies, and these jobs are similar to typical entry-level 
positions. These job postings identified 23 knowledge 
and skill items in three distinct categories: general IT 
knowledge and concepts, specific software package 
skills (e.g., Excel or Oracle), and general knowledge and 
skills applicable to any job. The job descriptions were 
then grouped by main job task, and 12 tasks emerged: 
analyst, database design, database modeling, database 
support, end-user training, help desk, network 
maintenance, software design, software development, 
technical writing, testing, and web design. 

It could be argued that job postings reflect what the 
employers think the universities can provide. However, 
the same job postings are generally sent to all 
universities in the area and many included skills that are 
not part of either IT program at our university. Thus, we 
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believe they are reasonably representative of what 
employers expect from Computer Science and MIS 
students and graduates in general. 

Three questionnaires were then created, one for each 
group. 

1 . entering students; 

2. exiting students; and 

3. IT graduates of the university. 

Some questions were common to all three question- 
naires, while others were aimed specifically at one 
respondent group. The common questions had to do 
with sources of knowledge about the IT marketplace, 
knowledge and skills, job tasks, starting salaries, and 
time spent interacting with people. A sample 
questionnaire is attached. A variety of methods were 
used to administer the questionnaires. Most subjects 
were asked to fill out a web survey, some were given or 
mailed a paper survey, and some were sent the survey 
over e-mail. In all cases, subjects were given the option 
of filling out a web survey, a paper survey, or an e-mail 
survey. In all but a few cases, those who responded 
preferred the web survey. 

Entering Students 

The questionnaire for entering students was 
administered to students registered in an introductory 
Computer Science course. This course was chosen 
because it is required for both MIS and Computer 
Science students and is usually taken in the first year of 
studies. Students taking the Computer Sciencecourse in 
the Fall 2000 and Spring 2001 semesters were asked via 
the class e-mail listand via personal visits to the class to 



fill out the survey on the web. They were given an 
information sheet about the survey, showing the address 
of the web survey, during the classroom visits. All 
students were given the option of filling out a paper 
survey, although none chose that option. 

Exiting Students 

The questionnaire for exiting students was given to 
students registered in two required capstone MIS 
courses. Students were contacted in the same way as the 
students in the Computer Science course. Several 
students filled out a paper version of the questionnaire. 
The survey was pretested in Fall 2000 and these four 
responses were included in the results. Exiting 
Computer Science students were not included. 

IT Graduates 

An e-mail was sent to the university’s alumni mailing 
list asking IT graduates to fill out a survey on the web. 
A similar e-mail was sent to selected MIS students who 
had graduated within the last three years. 

The response rate, summarized in Table 1, was only 
7.5% for entering students, while the response rate for 
exiting students was 72.2%. The low response rate for 
entering students was possibly due to the low amount of 
contact with the students. Despite the paper reminder 
handed to them in class, students had to remember to 
access the survey later after class. There was only one 
e-mail reminder a day later. However, it should also be 
noted that not all students in this class intended to 
complete an IT degree. Some other programs require 
this course and other students take it out of interest. 



TABLE 1 

SAMPLE SIZE AND RESPONSE RATE 



Targeted Group 


Invitations to 
Respond Sent 


Surveys 

Received 


Response Rate 
(percent) 


Entering Students 


2U0 


15 


7.5 


Exiting Students 


54 


w 


72.2 


IT Graduates 


57+*“ 


IT 


unknown 



* In addition to these 67 graduates , there were an unknown number of IT graduates 
belonging to the university’s 203-member alumni e mail list. 



162 



Proceedings of the I6 h Annual Conference of the International Academy for Information Management 



Thus, the response rate from the intended population is 
higher than the figure shown. 

The high response rate for exiting students was due to 
the support from the instructors, who gave their students 
several reminders to complete the survey. One 
instructor provided class time to administer paper 
surveys. The response rate for graduates is reasonable, 
given that there is no way of knowing how many would 
have received the e-mail request. 

RESULTS AND DISCUSSION 

Student Profile and Demographics 

Because of the small number of surveys received from 
entering students, all the student surveys are combined 
in this study. Their education and degree aspirations are 
summarized in Table 2. 



TABLE 2 

STUDENTS’ EDUCATION, 
DEGREE, AND MAJOR 



Percentage 


Highest Achieved Education 




High School 


48.1 


Diploma 


31.5 


Bachelor’s Degree 


18.5 


Degree Expected 




Bachelor of Management 


64.8 


Bachelor of Computer Science 


24.1 


Other 


11.2 


Major 




MIS 


63.2 


Computer Science 


22.4 


Other 


14.2 



The high percentage of students who already have a 
degree could be due to the job market and other factors. 
There were 54 student responses, 15 from first year 
students and 39 from third and fourth year students at 
the university. Of those responding to this question, 36 
were male and 1 7 were female. The age range was from 
18 to 44 (with an average of 25). The typical student 
who responded is working on a Bachelor of 
Management degree with a Management Information 
Systemsmajor; expects to make from $40,000 - $45,000 
CDN in the first job after graduation; expects to be 



working as an analyst, in database support, or in web 
design; and expects to be spending three to four hours a 
day working with people. 

Graduate Profile and Demographics 

There were 22 recent graduates who responded, 13 male 
and 9 female. They ranged in age from 23 to 43, 
averaging 29. Their education is summarized in Table 3. 

TABLE 3 

GRADUATES’ DEGREE AND MAJOR 



Percentage 


Degree 




Bachelor of Management 


60.0 


Bachelor of Computer Science 


30.0 


Other 


10.0 


Major 




MIS 


61.2 


Computer Science 


22.4 


Other 


16.3 



Graduates reported having taken anywhere from 5 to 30 
IT courses, with an average of 15. Those reporting in 
the upper range presumably included courses taken after 
graduation, since the MIS major requires seven IT 
courses and the Computer Science major requires 13. 
There were 20 graduates working in the IT field while 
two were not. Those working in IT worked anywhere 
from one to more than five years in the field; their 
responses were fairly evenly spread out over the ranges 
given. 

Sources of Knowledge About the Information 
Technology Market 

Students were asked to rank the sources of knowledge 
about the job market that they used. Newspapers were 
the top source, followed by work term position postings, 
other job ads/postings, fellow students, professors, and 
IT contacts. No students chose job interviews or high 
school counselors as their number one source. 
Apparently students did not maintain contact with their 
high school counselors once they entered university. 
The Internet was not listed as an option, so “job ads and 
postings could include both general job and company 
web sites along with other sources. In contrast, 
graduates chose the Internet as the source they use most 
often, followed by job postings, newspapers, and IT 
contacts. 
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Knowledge and Skills Used in an Information 
Technology-Related Job 

Students and graduates were given a list of 23 
knowledge and skill items which were divided roughly 
into general skills (applicable to any job), specific IT 
knowledge and skills, and knowledge of how to use 
specific software packages. They were to rank each' 
skill or knowledge on a 7-point scale as to its 
importance, with 7 being “extremely important and 1 
being “not important at all. 

General Knowledge and Skills 

General knowledge and skills are seen by both students 
and graduates to be fundamental in information 
technology-related jobs. The vast majority ranked all of 
the general knowledge and skills as “important, “very 
important, or “extremely important. The most impor- 
tant of these skills was problem solving, chosen by 
90.7% of students and 90.9% of graduates as extremely 
important or very important. The only statistically 
significant difference between students and graduates 
was that students rated teamwork as more important (at 
the 0.10 level using a t-test), perhaps reflecting the 
emphasis on teams in most of the courses. 

IT Knowledge and Skills 

Table 5 shows how students and graduates rated IT 
knowledge and skills. Both graduates and students felt 
that software development was the most important 
specific IT knowledge/skill, followed by systems design. 



They listed artificial intelligence and computer repair as 
not important at all or not very important. Graduates 
and students ranked IT knowledge and skills in 
approximately the same order, with students giving 
slightly more importance to each item than graduates. 
Students ranked installation/setup lower than graduates 
and end-user training higher than graduates. The only 
statistically significant difference was for artificial 
intelligence, at the 0.01 level, but this result is not very 
useful due to its low perceived importance by both 
groups. 

Knowledge of Specific Software Packages 

Table 6 shows that graduates and students both had 
similar rankings of software knowledge. The mean 
scores ranged from j ust under 4 (“somewhat important ) 
tojust under 6 (“very important ). The highest mean for 
students was database knowledge (5.87) while database 
knowledge and operating systems tied for the highest 
mean for graduates, at 5.73. Students generally gave 
more importance to all of the software skills than 
graduates, but only graphics was statistically significant 
(0.01 level). However, most graduates and students felt 
that knowledge of graphics was not important at all or 
not very important. 

University Teaching Effectiveness 

Exiting students and graduates were asked to rate the 
university’s teaching of the 23 knowledge and skill 
items identified above. They could choose any of seven 



TABLE 4 

RATINGS OF GENERAL KNOWLEDGE AND SKILLS 





Graduates 


Students 


General 

Knowledge & Skills 


“Extremely Important” or 
“Very/ Important” (%) 


Mean* 


“Extremely Important” 
or “Very Important” (%) 


Mean* 


Problem-solving 




6.64 


903 


6.44 


Communication 


80 


08 


813 


6.07 


Interpersonal 


63.6 


5.77 


81.5 


6.02 


Teamwork 


59.1 


537 


SO 


6.20 


Project management 


5015 


5 . 45 ~ 


0S3 


5.91 


Presentation 


353 


0 O 


30 


5.50 



* 1 -not important at all; 2-not very important; 3-only a little important; 4-somewhat important; 5-important; 



6-very important; 7-extremely important 
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TABLE 5 

RATINGS OF IT KNOWLEDGE AND SKILLS 





Graduates 


Students 


IT Knowledge 
& Skills 


“Extremely 
Important” or “Very 
Important” (%) 


Mean* 


“Extremely 
Important” or “Very 
Important”(%) 


Mean* 


Software development 


63.7 


5.73 


75.9 


5.93 


Systems design 


54.6 


5.55 


64.8 


5.56 


Testing 


54.5 


5.32 


50.0 


5.24 


Database design 


50.0 


5.36 


61.1 


5.59 


Network operating 


36.4 


5.05 


46.3 


5.35 


Data modeling 


31.8 


5.09 


50.0 


5.39 


Web development 


31.8 


5.09 


44.5 


5.06 


Technical writing 


27.3 


4.73 


37.1 


4.93 


Installation7set-up 


27.3 


4.64 


16.7 


4.83 


End-user training 


18.1 


4.86 


44.5 


5.22 


Computer repair 


0.0 


3.77 


11.1 


3.98 


Artificial intelligence 


0.0 


3.09 


11.2 


4.06 


* 1-not important at all 


; 2-not very important; 3-only a 


little important; 4-somewhat important; 5- 



important; 6-very important; 7-extremely important 



TABLE 6 

RATINGS OF SOFTWARE KNOWLEDGE 





Graduates 


Students 


Knowledge of Specific 
Software Packages 


“Extremely Important” 
or “Very Important” 
(%> 


Mean* 


“Extremely Important” 
or “Very Important” 
(%> 


Mean* 


Database 


59.1 


5.73 


66.7 


5.87 


Operating systems 


54.5 


5.73 


61.1 


5.70 


Internet applications 


40.9 


5.05 


38.9 


5.09 


Standard software 


36.4 


5.09 


48.1 


5.44 


Graphics/multimedia 


9.0 


3.68 


22.2 


4.69 


* 1-not important at all; 2-not very important; 3-on 


y a little important; 4-somewhat important; 5- 



important; 6-very important; 7-extremely important 



responses for each item, from 1, “has not taught me 
anything to 7, “has taught me a great deal. The first 
year students were not given this question, as they did 
not have the experience to answer it. 

General Skills 

Only 40.9% of graduates and 46.2% of students felt the 
university taught them a great deal or a lot about 
problemsolving. This percentage seems low, especially 
since problem solving was by far the most important 
skill identified by both groups. However, the means 
were 5.23 and 5.36 respectively for graduates and 
students. Graduates’ scores were generally lower than 



students’ scores, possibly because they have learned 
much more since they graduated. More students than 
graduates felt the university taught them a great deal or 
a lot about teamwork (69.3%), and the mean is 
significantly greater (0.02 level). 

Specific IT Knowledge and Skills 

Graduates and students felt the university taught them a 
great deal or a lot about database design, systems 
design, and database modeling. For all the other IT 
knowledge and skill items, most graduates and students 
felt the university did not teach them anything or very 
little. Only ratings for web design were statistically 
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TABLE 7 

RATINGS OF GENERAL KNOWLEDGE AND SKILLS LEARNED 





Graduates 


Students 


General 

Knowledge & Skills 


“Has taught me a 
great deal” or “Has 
taught me a lot” (%) 


Mean* 


“Has taught me a 
great deal” or “Has 
taught me a lot” (%) 


Mean* 


Problem solving 


40.9 


5.23 


46.2 


5.36 


Presentation 


40.9 


4.82 


48.7 


5.18 


Teamwork 


31.8 


4.82 


69.3 


5.72 


Communication 


31.8 


4.95 


38.4 


4.87 


Project management 


27.3 


4.23 


46.1 


4.85 


Interpersonal 


24.7 


4.23 


23.0 


4.49 



* 1-has not taught me anything; 2-has taught me very little; 3-has taught me a little; 4-has taught me 



something; 5 -has taught me a fair amount; 6-has taught me a lot; 7-has taught me a great deal 



significant (0.01 level), probably reflecting changes in 
the curriculum made in recent years. 

Knowledge of Specific Software Packages 

Students felt the university did a good job teaching them 
database software, standard software, Internet 
applications, and graphics/multimedia. Graduates 
agreed the university did a good job teaching database 
software and standard software, but their ratings on 
Internet applications varied greatly (probably depending 
on their graduation date). Students and graduates felt 
the university did not do a good job teaching them 
operating systems, probably because university 
programs are generally limited to PC environments 
using one standard (currently Windows 98). The 
differences were statistically significant for graphics 
(0.10 level) and database software (0.02 level). Students 
rated these two areas higher than graduates. 

Knowledge and Skills Needed vs Evaluation of 
University Teaching 

Table 1 0 provides a framework for comparing the skills 
and knowledge needed to those which are taught. Of 
most interest are the four labeled comers: 

1. The skill or knowledge is important, but the 
university did not teach much about it (attention 
needed). 

2. The skill or knowledge is important and well taught 
(good). 



3. The skill or knowledge was not important, and not 
taught (appropriate). 

4. The skill or knowledge was not important, but was 
emphasized in teaching (wasted resources). 

The skills and knowledge that exiting students and 
graduates considered the most important generally fell 
in area 2, i.e. problem solving, communication, 
teamwork, presentation, interpersonal, systems design, 
and database software. The two exceptions were 
software development and operating systems software, 
where the responses fell in areas 1, 2, and outside the 
four areas. This means there was not agreement among 
respondents as to importance and how well the 
university taugjit them. 

Type of Work in First IT-Related Job 

Students were asked to choose up to three job tasks that 
they expected to perform in their first IT-related job. 
They most frequently chose analyst (24), database 
support (22), and web design (19). End-user training 
and technical writing were the least frequently chosen 
responses. 

Graduates were asked to check off as many job tasks as 
were contained in their current job. Analyst was the 
most frequently selected task (9), followed by software 
development (6), end-user training (6), and web design 
(6). Database was selected only once, suggesting that 
graduates do not start in that area as often as students 
suspect. The lack of students selecting help desk tasks is 
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TABLE 8 

RATINGS OF IT KNOWLEDGE AND SKILLS LEARNED 





Graduates 


Students 


IT Knowledge 
& Skills 


“Has taught me a 
great deal” or “Has 
taught me a lot” (%) 


Mean 


“Has taught me a great 
deal” or “Has taught me 
a lot” (%) 


Mean 


Database design 


40.9 


4.86 


51.3 


5.26 


Systems design 


36.3 


4.91 


41.0 


4.85 


Database modeling 


27.2 


4.50 


43.6 


4.97 


Software development 


22.7 


3.91 


33.3 


4.38 


End-user training 


22.7 


3.45 


20.5 


3.74 


Testing 


4.5 


2.95 


2.6 


2.79 


Technical writing 


4.5 


2.95 


15.4 


3.69 


Network operating 
systems 


0.00 


2.59 


15.4 


2.97 


Web development 


0.00 


2.41 


5.2 


3.38 


Artificial intelligence 


0.00 


2.18 


5.1 


2.46 


Installation/set-up 


0.00 


2.14 


0.00 


2.18 


Computer repair 


0.00 


1.59 


0.00 


1.59 


* 1-has not taught me anything; 2-has taug' 


it me very little; 3-has taught me a little; 4-has taught me 



something; 5-has taught me a fair amount; 6-has taught me a lot; 7-has taught me a great deal 



TABLE 9 

RATINGS OF SOFTWARE PACKAGES LEARNED 





Graduates 


Students 


IT Knowledge 
& Skills 


“Has taught me a 
great deal” or “Has 
taught me a lot” 

(%) 


Mean 


“Has taught me 
a great deal” or 
“Has taught me 

-a Jot” -(-%) 


Mean 


Internet applications 


27.2 


4.09 


41.0 


4.72 


Standard software 


22.7 


4.55 


48.7 


5.18 


Database 


18.2 


4.36 


51.3 


5.36 


Graphics/multimedia 


18.2 


3.36 


35.9 


4.36 


Operating systems 


4.5 


3.64 


15.4 


3.46 



* 1 -has not taught me anything; 2-has taught me very little; 3-has taught me a little; 4-has taught 
me something; 5-has taught me a fair amount; 6-has taught me a lot; 7-has taught me a great 
deal r 
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TABLE 10 

IMPORTANCE OF SKILL VS AMOUNT TAUGHT BY UNIVERSITY 



Amount Taught by the 
University (1 = has not 
taught me anything, 

7 = has taught me a 
great deal) 



4. Wasted 
Resources 



2. Good 



L 



3. Appropriate 



1. Attention Needed 



1 


2 


3 


4 


5 


6 


7 


Importance of Skill or Knowledge (1 = not important at 


all, 7 = i 


extremely important) 









also interesting; while few want to work there it is a 
common entry point. 

Expected Salary 

Students were asked how much they expected to make 
in their first IT-related job after graduation. The 
students’ responses were clustered around the three 
middle ranges, from $30,000 to $44,999 CDN. 
Graduates were asked how much IT graduates should 
make in their first job after graduation. The majority 
chose $35,000 to $39,999 CDN. Mean salary 
expectations showed a downward trend from entering 
students ($40,625), to exiting students ($37,434), to 
graduates ($34,772). This could be related to their 
increasing experience as they progressed through their 
university program and then joined the work force. 

Graduates were asked if salaries in general met with 
their expectations. They reported that actual salaries 
were about the same as they expected or less than they 
expected. 



TABLE 11 

GRADUATES’ SALARY EXPECTATIONS 



Salary Expectations 


No. of 
Responses 


Actual salaries the same as expected 


9 


Actual salaries less than expected 


7 


Actual salaries more than expected 


4 


Don’t know 


2 



CONCLUSIONS 



This study showed a surprising level of consistency 
between the reality, as represented by recent graduates, 
and the expectations of students. Students recognized 
the importance of general skills in entry-level jobs and 
placed the highest importance onproblem solving skills. 
General knowledge and skills, such as problem solving, 
communication, interpersonal skills, and teamwork, 
continue to be important in entry-level IT jobs. 
Software skills are also important, such as knowledge of 
operating systems and database applications. One area 
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of difference was that students had higher salary 
expectations than graduates. 

Students and graduates felt the university is doing a 
good job teaching them the general skills. However, 
their perception was that the university did not do as 
well teaching them IT knowledge and skills. Whether or 
not this is true, the instructors of IT courses have an 
opportunity to tell students about the IT skills they are 
teaching and how they will fit into their careers, over 
both the shortterm and long term. 

The information technology field is growing and 
changing and information technology curriculum 
remains a moving target for educators. A continuing 
survey of IT graduates of the university would give a 
more accurate picture of the trends. What kind of jobs 
are they competing for and what kind of jobs are they 
getting? What skills and knowledge are important at 
different times in their careers? Such a long-term study 
would give university faculty up-to-date information 
about the marketplace and they would be in a good 
position to develop a solid IT curriculum and to create 
linkages with employers. Employers would have access 
to better trained employees over the long term and 
would have better knowledge of graduates’ skills and 
knowledge. 
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APPENDIX 

SURVEY OF INFORMATION TECHNOLOGY STUDENTS 
ENTERING STUDENTS 



You are asked to complete and return this survey. The information from the survey will enable me to learn more 
about the students who choose Information Technology (IT) majors at the University of Lethbridge. Participation 
in the survey is voluntary and your responses will be kept strictly confidential. Your ID number will be used only 
to ensure each completed questionnaire is unique, and will be strippedfrom the questionnaire once all the responses 
have been received. The information will be reported in general terms without specific reference to individual 
responses. 

If you have any problems with this form, please e-mail hodd@uleth.ca 



ID Number: 



1 . What sources do you use to find information about the Information Technology (IT) market? Rank each source 
you use. 



Professors 
Fellow students 
Job interviews 
Job ads and postings 
Co-op position 



Contact(s) in the IT industry 
Newspapers, magazine articles, on-line 
articles 

High school counselors 

Other specify 



2. How much do you know about the MIS major (Bachelor of Management program)? Place an X beside the 
appropriate answer. 





I know nothing about it 




I know almost nothing about it 




I know very little about it 




I know a little about it 




I know something about it 




I know a fair amount about it 




I know a lot about it 



3. How much do you know about the Computer Science major (Bachelor of Science program)? Place an X beside 
the appropriate answer. 





I know nothing about it 




I know almost nothing about it 




I know very little about it 




I know a little about it 




I know something about it 




I know a fair amount about it 




I know a lot about it 
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4. Knowledge and skills that you might use in an Information Technology-related job are listed below. How 
important do you think each will be for your first IT-related job after graduation? Write a number beside each 
item to indicate its importance. 

1 2 3 4 5 6 7 

not important not very only a little somewhat very extremely 

at all important important important important important important 



communication skills (oral & written) 

interpersonal skills 

presentation skills 

problem-solving 

team work 

project management 

technical writing 

end-user training 

artificial intelligence 

computer repair 

data modeling 

database design 

network operating systems 

systems design 

software and hardware testing 
software and hardware installation and set- 
up 



web development (e.g. html, JavaScript, 
vbscript, PERL, or Active X) 
software development (e.g. Java, Visual 
Basic, MS Access, or Visual C/C++) 
knowledge of operating systems (e.g. DOS, 
Windows, Unix, or Linux) 
basic knowledge of standard software (e.g. 
word processing, spreadsheet, or 
presentation) 

knowledge of internet applications (e.g. 
Netscape, Internet Explorer, or Front Page) 
knowledge of graphics and multimedia 
applications (e.g. Paint Shop Pro, Adobe 
Photoshop, or Adobe Illustrator) 
knowledge of databases (e.g. MS Access, or 
Oracle) 



5. In any job, you will spend sane time working at the computer on your own and some time interacting with 
people (conferring with your workmates, attending meetings, and talking on the telephone). How many 
hours per day on average do you expect to spend interacting with people in your first IT-related job? Choose 
one response. 

less than 1 hour 1 - 2 hours 3-4 hours 5 - 6 hours more than 7 hours 

6. What type of work do you expect to be doing as your primary task in your first IT-related job? Choose up to 
three items. 



analyst 


software development 


database design 


testing 


data modeling 


technical writing 


database support 


end-user training 


network maintenance and 


user support/help desk 


installation 


web design 


software design 





7. What salary do you expect to receive in your first IT-relatedjob after graduation (in $ CDN)? 



$20,000 - $24,999 $25,000 - $29,999 $30,000 - $34,999 

$35,000 - $39,999 $40,000 - $44,999 $45,000 - $49,999 

$50,000 and above 




Proceedings of the 16 h Annual Conference of the International Academy for Information Management 

BEST COPY AVAILABLE 



171 



187 



Demographic Information 

8. Sex M F 



9. Age 

10. Highest level of education achieved to date: 

High School Diploma College Certificate College Diploma 

Bachelor’s Degree Master’s Degree 

1 1. What degree(s) do you expect to receive when you graduate in your currently registered program? 

Bachelor of Management Bachelor of Education 

Bachelor of Science Bachelor of Arts 

combined degree in Management and Education combined degree in Arts and 

Management 

combined degree in Science and Management other 

If you checked “other above, what degree/diploma/certificate will you receive when you graduate? 



12. What is your first major? 

13. How many courses do you have left until you graduate, not including the current semester? (Do not count co-op 
work terms.) 



14. How many courses have you taken already, including the current semester? (Include courses taken at other 
colleges and universities and exclude co-op work terms.) 



15. How many IT-related courses have you taken so far, including the current semester? (Include IT courses taken 
at other colleges and universities and exclude co-op work terms.) 



Thank you for your help! 

Paper survey: Please return the survey in University of Lethbridge intercampus mail to: 

Pat Hodd 

Room E-480, Faculty of Management 
The University of Lethbridge 
4401 University Drive 
Lethbridge, AB T1K3M4 

Web survey: Fill in the survey found at http://home.uleth.ca/~hoddTTstudents/surveyl/ and click on the 

“Submit button. 

If you would like to receive the results of this survey, they will be posted on the following web site: 

http://home.ulah.ca/~hodd/ITstudents 
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ABSTRACT 

The application of Problem Based Learning (PBL) approach to learning and teaching is becoming more and more 
popular in many areas of professional Education including Medicine, Chemistry, Biology, Law, and Business 
Administration. Recently some teachers in the area of Information Systems education began to use this strategy. One 
of the major problems and concerns of both educators and students in the PBL setting is the assessment of studen t ’ 
work. In this paper, the authors have followed up their previous work and present their approach to students’ 
assessment in the information systems courses. In our approach, we measure several students ’ skills such as critical 
thinking, communication skills and so forth as well as doing some traditional assessment such as tests and quizzes 
with a lower percentage than traditional. Students’ perceptions of our assessment strategy have been evaluated 
through the end of semester survey . It is the authors ’plan to make the assessment strategy more general with the 
hope that other instructors are able to use it in their own courses. The authors have taught an information system 
course in their respective institutions using PBL as a teaching strategy as well as applying the assessment strategy 
to assess students ’ work. The outcome of the application of PBL, the assessment strategy, and students ’ perception 
of PBL assessment are presented in this paper. 



INTRODUCTION 

Problem-based learning (PBL) isan educationalstrategy 
that uses problems as the starting point for student 
learning (Bligh, 1995). It is a curriculum design and 
teaching/leaming strategy, which recognizes theneed to 
develop problem-solving skills as well as the necessity 
of helping students to acquire the necessary knowledge 
and skills (Boud & Feletti, 1997; Biggs 1999). The main 
issue is to reduce direct instruction as students assume 
greaterresponsibility fortheir own learning. Students are 
given ill-structured problems through which they 
develop high-order thinking and problem-solving skills. 
The shift in the teaching and learning process is more 
student-centered than teacher-centered. The role of the 
teacher is to encourage student participation, provide 
guidance to students, offertimely feedback, andassume 
the role of learner as well (Aspy et al., 1993). Evidence 



of the success of problem-based learning as an 
instructional strategy is strongly positive, particularly in 
fostering increased knowledge retention (Norman & 
Schmidt, 1992), encouraging general problem-solving 
skills, for deep-biased students (Norman & Schmidt, 
1992; Lai, Tiwari & Tse, 1997 ), promoting self-directed 
learning skills, and increasing intrinsic interest in the 
subject matter. Since its introduction as an instructional 
method used in the medical school at McMaster 
University in Ontario in the 1960s, PBL has spread to 
numerous educational institutions around the world. To 
the best of our knowledge, the PBL strategy has not 
become so popular in the field of computer science and 
information systems. We have applied the methodology 
to our respective institutions for the last couple of years 
and it proved to be successful. The course that we have 
applied is- Systems Analysis and Design. However, 
students’ assessment often proves to be difficult for 
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teachers. The traditional assessment using multiple 
choice or true and false tests do not seem to be suitable 
for PBL approach to teaching. As students work in 
teams, there is a concern of assessing their contribution 
in teamwork and individual performance, as well as 
other skills. That is why assessment instrument need to 
be created for PBL approach. 

Woods (1996) presents 16 issues related to PBL 
assessment. His suggestion is more of theoretical ideas 
about measuring students learning process. There are no 
specific guidelines on the way that some or all of these 
issues can be applied in practice. They are all based on 
Bloom’s taxonomy (1956). It is not clear whether we 
should develop separate criteria for PBL assessment, use 
traditional tests and quizzes, or a combination of both. 

Newak and plucker (1999) suggest that the PBL 
instruction and assessment should be aligned. They 
proposed four suggestions for this alignm ent. Motiv ate 
students, create hands-on-learning environment, provide 
reasonable guidelines, and model real-world behavior. 

Hicks (1998) suggests that PBL assessment at times can 
be a formidable task but can be done in different forms 
such as teachers observation, written material and 
products, peer evaluation, self-evaluation, and feedback 
from the outside community. 

The subject of this paper is the application of the 
authors’ PBL assessment instrument that has been 
developed in their previous paper. This instrument 
contains a set of criteria that are focused in nine different 
areas namely, Individual work, Teamwork, Creative 
Thinking, Analytical Skills, Communications skills, 
Leadership skills, and course assessment. Students’ 
perceptions of this assessment have been performed 
through regular end of semester survey and the results 
proved to be very favorable. The students of the two 
institutions have marginally different perceptions with 
regard to PBL assessment. Nevertheless, the majority of 
our respective students favor our assessment as opposed 
to traditional assessment. In the following sections we 
present the details of the assessment strategy. 

THE ADOPTED PBL STRATEGY 

When adopting the PBL strategy for teaching, 
assessment of students work is one of the major 
concerns. Why is assessment important? Assessment can 
drive students to work. Also, students learn what is 
assessed (Woods 1996.) In an attempt to solve this 



problem, the authors have developed an assessment 
instrument for the information systems courses, 
specifically Systems Analysis and Design and then 
applied the instrument in their respective institutions. In 
deciding what to assess students using PBL approach to 
learning and teaching, we posed the question, initially, 
of what should be the qualities of IS graduates and we 
considered the following skills that graduates should be 
able to: 

1. Work independently 

2. Work teams 

3. Think critically in problem-solving 

4. Have analytical thinking 

5. Communicate (both oral and written) 

6. Have leadership potential 

We believe using these skills as assessment of student 
work is much more effective than requiring them to 
memorize the lecture topics, take tests, and later forget 
about them what they have learnt. However, we have 
attempted to address the above skills as follows. 

1. In the PBL setting students are generally given a 
problem to solve. In the case of Systems Analysis 
and Design they are given a problem to analyze and 
design. The problem is not well defined. In another 
word, the detail of the specification is not provided 
to the students. As a result students must choose 
their own way of learning and to come up with a 
solution strategy for the problem. For example, they 
may read their books, search the Internet, go to the 
library and re ad some other books o r peri odicals . All 
these activities are designed to develop positive 
attitude towards learning, organize their work and 
behave professionally. 

2. Every IS professional needs to be compatible with 
people. Teamwork is an important part of the IS 
profession. Students need to be trained to respect 
and listen to other people’s opinions. In teamwork 
situation, they need to distribute work fairly among 
the team members. Students need to attend group 
meetings and meeting the project’s deadline. In 
addition students develop some ethical behavior. 

3. Critical thinking is a state of mind that is not 
achieved automatically. Students must be trained. 
Students must be trained to think critically towards 
solving a problem. We believe that PBL strategy can 
help students to gain critical thinking skills. This 
activity is designed for students to propose creative 
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ideas, viewing at other people’s work critically 
and justify their solutions, which in turn help 
them to stretch their mental ability. 

4. Students need to possess analytical and problem- 
solving ability. In many situations students are 
required to discuss and cope with ambiguous 
problems. In this activity students will practice and 
learn approaches to redefinition of their problem in 
an understandable manner. They learn how to 
approach in solving a problem. For example, 
decomposing the problem into pieces and assign 
each piece to one of the team members. At their next 
meeting they share their solution with each other. 

5. Communication is another skill that is very 
important for an IS professional. There are many 
levels of communications namely; communications 
with team members, communication with project 
manager, and communication with clients. In 
addition, in this activity students will also practice 
written communications through their project 
submission and also oral communication by 
presentation their projects . As a result students will 
develop communication skills. 

6. Leadership is an, integral part of IS professional. 
The project leader is responsible to meet the 
deadline, to monitor and control the project, 
decision-making, and so forth. In the traditional 
teaching, students hardly have a chance to practice 
these issues whereas in the PBL setting due to its 
nature the leader must control the project. 

STUDENTS’ WORK 
ASSESSMENT IN THE PBL SETTING 

In this section we will explain the details of the 
application of assessment instrument in our courses. 
Basically the assessment of students’ work was done in 
three steps as described below. 

Step 1 

In this step we followed their learning process very 
closely throughout the course. The students’ work in 
each project phase has been monitored and compared 
with the previous phase. This is to ensure that the work 
will be good quality though out. We are also concerned 
that they can integrate knowledge to practice and their 
reports have been written andpresented in a professional 
manner. Should students’ work are in need ofchanges or 



modifications, they will be informed with 
recommendations and suggestions to resubmit their 
work. In this step we do actually more than just grading 
of the students work, students have a chance of reflecting 
on their work performance. 

Step 2 

In this step the evaluation is done based on the students 
report of each phase of their project. The following is a 
list of the items that students are required to submit 
along with their projects. 

• Log of their work 

• Amount of time spent reading books and the page 
numbers that they read 

• Amount of time spent working on the internet, 
submitting the URL they used and the subject they 
studied 

• Amount of time spent communicating with team 
members 

• Amount of time spent on meeting for a particular 
phase of the project including the topic that was 
discussed on that meeting 

• Problems that they were encountered during the 
meeting 

• Peer review for each of the project 

• For presentation of their project we measured 

• Quality of their presentation 

• Use of time 

• Eye contact 

• Use of visual aid 

• Answer to questions 

• Fluency of the topics being related to each other 

• Confidence on the subject matter 

Step 3 

To accommodate the needs of students with different 
learning styles we also performed some traditional 
assessment through tests and quizzes. The percentage of 
this component is about thirty percent of their total 
grade. Some of the students who were willing to work 
but were having difficulties intheir projectthrough PBL 
strategy could focus on this aspect of evaluation. The 
other reason for having traditional assessment 
component is the following. Students are used to 
traditional assessment all their study life in many other 
courses. Suddenly changing that habit in our courses 
make them feel uncomfortable and are concerned about 
their grade. Adding this component will reduce this 
anxiety. 
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STUDENTS’ PERCEPTION 
OF THE PBL ASSESSMENT 

In Fall 2000 the authors both applied the PBL strategy in 
their Systems Analysis and Design courses. The PBL 
assessment instrument that we developed and was 
presented in the last section have also been applied to 
our their respective courses. We found this experience 
very motivating to students and instructors. The most 



interesting part was the students’ interaction with each 
other through team meetings and other ways of 
communications. To see students’ perception of our 
approach we conducted an end of the semester survey 
concentrating on different learning issues as compared 
with traditional teaching methods. A total of 46 
questions were asked in the survey addressing different 
issues in the course. The survey results are depicted in 
the following charts. 



FIGURE 1 

WRITTEN TESTS AND QUIZZES 

The percentage of written test and quizzes should 
carry in this course 
(Comparison of HK and US students) 



03 HK 

□ US 



0. 00* 10. 00* 20. 00ft 30.00% 40.00% 

Percent age 




FIGURE 2 

INDIVIDUAL WORK 



Individual fork (Comparison of HK and DS students) 




Disagree Ho Ac ree Strongly 

Difference Acre* 
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FIGURE 3 
TEAMWORK 



Team fork (Comparison of HK and US students) 



Ethical behavior in croup 
envi ronment 



Attending regular team 
meeting 



Keetinc the deadlines 



Skills of transferrinc 
knowledge to others 




Q HK 
□ US 



FIGURE 4 

CREATIVE THINKING 



Creative Thinking (Comparison of HK and US students) 



Generating creative ideas 



Making good 
justi fi cati ons 

Offering solutions to a 
problem 



Applying knowledge to 
practi ce 

Mental ability stretch 



Critical thinking skills 



2 3 4 5 

Disagree No Difference Agree Strongly 

Agree 
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FIGURE 5 

ANALYTICAL SKILLS 



Analytical Skills (Comparison of HK and US students) 



Problem analysis skills 



Problem definition skills 



Approaches to solve a 
problem 

Pattern recognition 
skills 

Skills to cope with 
ambigui ty 



Problem solving skills 



Reasoning skills 



2 3 4 5 

Disagree Ho Difference Agree Strongly Agree 




FIGURE 6 

COMMUNICATION SKILLS 



Communication Skills (Comparison of HK and US students) 



Clarity of written 
presentati on 



Fundamental interpersonal 
skills 



Presentation skills 



Report writing skills 



Written communication 
skills 




Q HK 
□ US 



5 

Strongly Agree 
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FIGURE 7 

LEADERSHIP SKILLS 



Leadership Shills (Comparison of HK and US students) 

Organi z ati onal skills 
Ethical behavior 

Assertiveness skills 

Self-assessment and peer 
assessment 

Decision-making skills 

Project controlling 
skills 

Project planning skills 
Leadership skills 

2 3 4 5 

Disagree No Difference Agree Strongly Agree 




FIGURE 8 

COURSE ASSESSMENTS 



Course Assessment (Comparison of HK and US students) 



Formal lecture should be replaced by more group 
work 




The lab/tutorial time should be more than what 
we had 

What percent should quizzes carry 
There should be quizzes in this course 
What percent should written test carry 
There should be a written test in this course 



3.27 




2 3 4 

Disagree No Difference Agree 



9 HK 
□ US 



5 

Strongly Agree 
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ANALYSIS OF THE RESULTS 

The charts depicted above demonstrate that the majority 
of students in both institutions agree that, in the PBL 
settings, they have developed skills far beyond what they 
would have done in traditional teaching settings. 
However, there are significant discrepancies in certain 
areas. For example, Use of library, searching on the 
Internet, life-long learning, and positive attitude towards 
learning skills developments are rated lower for the US 
students compared with the students in Hong Kong 
school of our study, see Figure 2. 

Students in Hong Kong school seem to be more 
comfortable with PB L setting than the US students. This 
reflected in their ranking of PBL assessment verses 
traditional assessment. The US students want more 
percentage for formal test and quizzes (average of 25 
percent). Whereas the Hong Kong students want lower 
percentage for formal tests and quizzes (average of 18 
percent.) see Figure 1 and Figure 8. This reflects that 
students in HongKong are more comfortable with PBL 
strategy and our assessment mechanism. This is due to 
the fact that the PBL strategy is more popular in the 
Hong Kong school. 

Nevertheless our observation is an isolatedstudy for two 
schools only. More formal studies are required to draw 
a general conclusion. However, our study proves that 
students in PBL strategy and its subsequent assessment 
are far more professionally active than when the 
traditional teaching is applied. Some of these activities 
include, commitment to teamwork and its 
responsibilities (See Figure 3), creative thinking (see 
Figure 4), analytical skills (see figure 5), communi- 
cations at different levels see Figure 6), and taking 
charge of the learning process instead of relying too 
much on the instructor (see Figure 7). 

CONCLUSION 

Evaluation of students’ achievement is an important 
aspect of education, and the skills required inherent in 
the process of solving real-world problems must be 
included in that assessment. The authors believe that 
alignment of instruction and assessment is essential 
when the PBL strategy is applied to our courses. In this 
study we focused on the generic ability of the IS 
graduates. The assessment instrument proved to be 
successful in many ways. It had significantly reduced 



students’ anxiety with regard to accuracy and 
methodology of assessment. It has created a relatively 
fair and easy to use methodology of assessment, which 
is the major concern of both students and instructors who 
apply the PBL strategy in their courses. The assessment 
works best for the courses that we have applied. This is 
because the organization of keeping the log of your work 
on a daily basis is part of the task that systems analysts 
and software developers do in practice. The PBL 
strategy had several positive results that are worth 
noticing. The rate of students learning activities 
including, reading, searching, and communication was 
much higher compared to the traditional teaching and 
assessment for the same courses. 
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ABSTRACT 

This paper reports findings from a study on using computer-based assessment (CBA) in a university course. Data 
gathering took place over two semesters and includes findings from over 130 students and three faculty members. 
The CBA tool used was an application-based testing product that measures students' proficiency in computer 
applications such as Microsoft Word, Excel, Access and PowerPoint 2000, while working live in the actual program 
application. The product, SAM2000 (Skills Assessment Manager) allows students to perform real-world tasks in a 
“live” testing environment. Data for this project was gathered through (1) questionnaires completed by students 
which evaluated their experiences and reactions to the computer-based tests as assessment tool including perceptions 
on ease of use, fairness of grading, perceived impartiality, perceived difficulty, and perceived level of performance; 
(2) informal interviews with faculty that administer computer-assisted tests to determine their perceptions of using 
this medium for testing — both advantages and disadvantages; and (3) analysis of student test scores in both 
conventional paper -based tests and computer-based assessments. 



INTRODUCTION 

The rapid growth of information technology is changing 
the way the world does business. Similarly, it is 
changing the way teaching and learning take place. As 
such, institutions of higher education must use 
technology in the presentation of course content and the 
assessment of academic abilities. Computer-based 
assessment (CBA) is becoming an essential component 
to higher education. 

Computers are used in every aspect of our lives, 
particularly in the business world. Since college 
students will no doubt be using computers in their 
careers, there is no question that course content should 
be taught using technology and computers. Therefore, 
assessment should be based on the same concepts and 
should be tested in the environment in which the 



concepts are used. CBA is becoming an essential 
componentto higher education. Furthermore, computers 
are beingusedto conduct large-scale graduate placement 
tests, such as the Graduate Record Examination (GRE) 
and the Graduate Management Admission Test (GMAT), 
professional certification exams, such as the Series 
Seven exam, vocational interest and aptitude assess- 
ments, and workplace skills tests. W ith such high-stake 
exams being offered on computers, students will be 
taking tests using this medium and faculty members need 
to prepare them . 

Computer-assisted testing provides a tool for formal 
assessment of large groups of students without 
overburdening teaching staff with test-grading 
responsibilities (Zakrzewski and Bull, 1998). If we 
expand to include the area of distance education where 
entire courses are delivered remotely via technologies 
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such as course websites, e-mail, Listservs, bulletin 
boards, and/or videos, it is imperative that assessment 
can be delivered remotely via computer-based testing. 

Given the importance of technology for assessment, 
various studies on implementing computer-based 
assessment have been conducted. Primarily, these 
studies focus on particular software packages used for 
delivering and grading “short-answer” exams with a 
computer. Some studies are based on implementations 
in elementary and m iddle schools (Russell, 1 999) while 
others studies focus on higher education (Bocij and 
Greasley, 1999 and Zakrzewski and Bull, 1998). 
Cutshall (2001) provides a discussion on implementing 
various formats of CBA in career and technical 
education environments. Thelwall (1999) provides an 
overview of the types of CBA tools. In particular, 
Thelwall discusses the benefits of implementing 
“random-based tests” where a set number of questions 
are random ly selected an d admin istered to an examinee 
from a large test bank of questions. 

This paper focuses on a study to assess the effectiveness 
of using “live” performance-based assessment in an 
undergraduate computer applications course. The 
overall goal of the research is to identify the impact of 
using live-in-the-application (live) testing software on 
student performance. Our primary goal is to evaluate if 
live testing is more valid and reliable than traditional 
testing methods. In addition, we are interested in 
investigating both student and faculty acceptanceof live 
testing in place of traditional paper-based assessment. 

Sam 2000, the CBA tool selected for this project, 
measures students' proficiency in Microsoft Office 2000 
including Microsoft Word, Excel, Access, and 
PowerPoint while working live in the actual program 
application. The CBA automatically grades the students’ 
responses based on the specific methods required to 
obtain the desired results. 

For the purpose of this project, we define the traditional 
paper-based assessment technique as a paper test 
consisting of instructions to complete a series of tasks 
while working in the actual software application being 
evaluated. Students are graded on their final product, 
not on the specific methods used to obtain the desired 
results. 



DATA GATHERING 

This study was conducted over two academ ic semesters 
in multiple sections of an introductory computer 
applicationscourse . All of the students in the study w ere 
administered two tests evaluating their proficiency in 
Microsoft Excel 2000. The first test was administered 
via Sam 2000; the second test was a comparable paper- 
based test assessing the same skills as the computer- 
based test. Scores for both tests were recorded and the 
students were given a follow-up questionnaire to 
evaluate their experiences using both mediums, 
particularly the CBA technique. Data for this project 
was gathered through: 



Recording Student Perceptions of Using 
Computer Based Assessments 

• participant observations 

• post-test survey 

• informal interviews following CBA 

Recording of Student Background Information 

• background questionnaire items related to educational 
background, familiarity with computers, and experi- 
ence with computer based examinations 

Recording Faculty Perceptions of CBA 

• informal interviews with faculty who administer 
computer-assistedtests to determine theirperceptions 
of using this medium fortesting — both advantages and 
disadvantages 

Recording of Examination and 
Assessment Results 

• direct capture by the CBA 

• analysis of detailed student performance reports 
generated by the CBA 

• analysis of student test scores to compare performance 
between traditional paper-based tests and computer- 
based assessments 



CBA Software Description 

This research focuses on one type of CBA within the 
family of skills-based testing software. Skills-based 
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products either test“live-in-the-application” or simulate 
the application environment. In both product groups, 
examinees are evaluated on their ability to perform 
specific tasks in the application being tested. In 
selecting the specific program used in this study, we 
evaluated several programs within this family, the 
majority of which were simulated. We selected SAM 
2000 (Skills Assessment Manager), available through 
Course Technology (www.course.com), which allows 
students to perform real-world tasks in a “live” testing 
enviro nment. 

SAM 2000 tests the user’s proficiency while the user 
works live in the application. For example, when a user 
is tested on formatting text in Microsoft Excel 2000, the 
user completes exactly the same steps they would if 
formatting the text outside the SAM 2000 environm ent. 
An instructor creates an exam to test skill proficiency in 
a specific application (e.g., Microsoft Word, Microsoft 
Excel, etc.). Questions are selected by the instructor 
based upon the particular skills that they want to test. 
When students take a SAM 2000 exam, each skill-based 
question is presented individually; students have the 
opportunity to redo any question on the exam provided 
that the allotted exam time has not expired. Upon 
completion of an exam, students are presented with 
reports on both their overall performance and detailed 
information on specific skills that were completed 
incorrectly with reference to related sections of their 
text. 

Study Group 

This study focuses on incorporating CBA techniques 
into an undergraduate computer skills course in the 
School of Business at Montclair State University, a 
regional four-year institution in northern New Jersey. 
The Computers in Business course is a required course 
for all undergraduate majors in the School of Business. 
The course is divided into three modules as follows: 
Module 1 — Word 2000; Module 2 — Excel 2000; and 
Module 3 — Access 2000. Our study, conducted over 
two semesters, examines eight sections of this course 
including both day and evening sections; three different 
instructors taught these sections. All sections selected 
for the study followed a similar syllabus, administrated 
comparable examinations, and receivedthe same number 
of contact hours. 

A total of 133 students participated in the study — 70 
students in the Fall 2000 semester and 63 students in the 
Spring 2001 semester. Of these students, there are 22 



freshmen, 45 sophomores, 48 juniors, and 1 8 seniors. Of 
them, 88% (1 16) are full-tim e students and 1 2% (17) are 
part-time students. The breakdown by student’s major 
is found in Figure 1 . 

FIGURE 1 

BREAKDOWN OF STUDENTS 
IN THESTUDYBY MAJOR 



Students by Major 




Of the 133 students in the study, 9 1 of thestudents have 
never taken an exam on a computer priorto enrolling in 
the Computers in Business course. The remaining 42 
students had taken at least one exam on a computer in 
high school or college, or has taken a standardized 
educational test such as the GRE. 

Construction of CBA and Paper-based Examinations 

During the course of the semester, each student had 
multiple exposures to taking CBA. Students’ first 
experience with the CBA was in the form of a three (3) 
question practice exam. This exam did not contribute 
towards the student’s course grade and students had the 
opportunity to take the practice exam multiple times to 
ensure that they were comfortable navigating the CBA 
environment. Instructors were available to address any 
technical problems and observe the reaction of students 
to the CBA environment. The next exposure to theCBA 
was their Module 1 examination covering Microsoft 
Word 2000 tasks; this exam was scored and contributed 
toward the student’s final grade. The CBA results 
reported in this study are derived from Module 2 
examination covering Microsoft Excel 2000 tasks. The 
Excel CBA exam was constructed by choosing twenty- 
five (25) questions related to skills learned in class 
during Module 2. The exam was created using an 
iterative process and involved instructors from all 
sections included in the study. 
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Student’s scores on the CBA were automatically 
recorded. Data recorded by the CBA includes overall 
score achieved, question-by -question performance 
analysis, and elapsed testing time. Grades on the CBA 
are based solely on the student’s ability to complete a 
specific skill-based operation in the application; in this 
case, the students are tested on Microsoft Excel skills. 
There are multiple ways in which each question can be 
answered and scored correctly; for instance, if a student 
is asked to make a title bold they can use keyboard 
shortcuts, the edit menu, or the toolbarto complete the 
task. There is no partial credit on the CBA. 

The paper-based exam ination was constructed to mirror 
the tasks tested in the CBA. As in the case of the CBA, 
the paper-based exam was created using an iterative 
process and involved instructors from all sections 
included in the study. Questions on the paper-based 
exam were phrased in a similar manner to those on the 
CBA. Each question on the paper-based exam directs 
the student to perform a task in Microsoft Excel. 
Students could perform tasks in any order and could 
revise their answers at any time provided that the 
examination is completed within the time limit. 

The paper-based examination was graded manually by 
viewing the printoutand actual Excel file containing the 
completed examination. Results of the paper-based 
exam were recorded by physically checking exam 
responses and noting the required information in an 
Excel spreadsheet. Grades on the paper-based 
examination are based on the final product submitted by 
each student, as opposed to how each task was 
performed. For example, a student could be asked to 
insert text in a cell using the Copy and Paste 
functions — in the paperbased exam the instructor would 
grade the student based on the appearance of the correct 
text in the correct cell ratherthan on HOW the text was 
inserted in the cell — on the CBA the student would be 
graded on HOW they inserted the text not on it 
appearance. Partial credit was awarded on the paper- 
based exam. 

RESULTS 

Student Perceptions 

A post-examination survey was used to gather 
information concerning students’ experiences using 
computer-based assessment software. Questionnaires 
were adapted from existing sources such as Miller and 



Engemann (2000) or Bocij and Greasley (1999) in order 
to improve validity and reliability. 

All data collected was prepared for analysis with 
spreadsheets and statistical packages. Eight questions 
were administered to examine student attitudes and 
perceptions toward computer-based assessment. The 
summaries that follow are based upon all student 
responses, participant observations, and informal 
interviews with students. 

The firstquestion evaluated students’ com fort with using 
the software. 

Ease of use. Students evaluated the ease of use in 
navigating the Sam 2000 software environment. 
Students rated the difficulty level of the mechanics of 
accessing SAM, taking the exam, and reviewing the 
exam results. As shown in Figure 2, the majority of the 
students (59%) found the software to be easy or very 
easy to use. While 29% found it to be moderate and 
only 12% found the software to be somewhat difficult to 
use. 

FIGURE 2 
EASE OF USE 



I found the mechanics of SAM to be... 
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Very Somewhat Moderate Easy VeryEasy No 
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By the time the surveys were completed the students had 
several experiences with the software. Forthe most part, 
students found the exam interface to be straightforward 
and user-friendly. 

The next pair of questions evaluated the quality of Sam 
2000 as an assessment tool by considering the range of 
knowledge assessed by this computerized examination 
process and the difficulty of the exam. 

Range of skills. This question asked students to compare 
the range of knowledge and skills assessed by the 
computer-based testing with actual course content. 
Figure 3 displays that the majority (76%) of the 
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respondents found that SAM effectively tested the full 
range of skills taught in the course. Nineteen percent of 
the respondents felt neutral or did not respond to the 
question. Only 5% of the respondents did not feel that 
the test effectively tested all the skills taught. A number 
of the students who disagreed with the statement 
indicated that they were not tested on all the skills 
presented in the class. None of the students indicated 
that they were evaluated on skills that had not been 
taught. 

FIGURE 3 
RANGE OF SKILLS 
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Question difficulty. Students evaluated the difficulty of 
test questions on the Sam 2000 exam. Figure 4 
demonstrates that the lai^e majority of the students 
(65%) felt that the questions were moderate, while 23% 
of the students found the questions to be either easy or 
very easy and 12% found the questions somewhat 
difficult. 



FIGURE 4 

QUESTION DIFFICULTY 
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adequately tested the skills that were taught in the 
course. 

Students were also surveyed on their opinions of 
computer-based testing as a fair method of assessment. 
These questions evaluated students’ perceptions on 
grading impartiality and impact on performance. 

Grading impartiality . This question ev aluated students ’ 
perception of the fairness of the automated grading 
system included in the software. As shown in Figure 5, 
34% of the respon dents felt that the grading in Sam 2000 
was fair while 39% of them did not find the grading fair. 
The remaining students (26%) were neutral. As 
mentioned previously, SAM did not provide partial 
credit to incorrect responses; therefore, students would 
receive no credit if they included an extra space while 
inserting text despite performing the correct actions. 

FIGURE 5 

GRADING IMPARTIALITY 
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Another recurring criticism from students was thatS AM 
did not provide capabilities for students to review their 
specific responses to each questionto see why they were 
graded incorrectly (i.e., students could not view each 
keystroke they performed.) 

Impact on performance. Students evaluated the effect 
of using a CBA tool on their exam performance. As 
shown in Figure 6, only 19% of the students felt that 
using Sam 2000 negatively impacted their performance 
on the exam. The remaining students felt neutral or 
disagreed with the fact that using SAM impacted their 
perform ance. 



The overall consensuswas that the students feltthat Sam 
2000 was a reasonable assessment tool and 
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FIGURE 6 

IMPACT ON PERFORMANCE 
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Students were being evaluated in the same environment 
in which the skills were learned. Although most students 
have never been tested on a com puter prior to the course, 
the majority of the students were comfortable being 
evaluated in this environment. 

The next group of questions asked students to compare 
computer-based assessment to conventional 
examinations and to provide their preference for a 
particular exam medium. 

Comparison to paper-based test. Students compared 
the difficulty of the CBA (Sam 2000) with the paper- 
based assessment. Figure 7 demonstrates that 37% 
found that the two tests were comparable, neither easier 
nor more difficult. Only 14% found that the SAM test 
was easier and the rem aining 49% found it more d ifficult 
or much more difficult. Two points should be noted. 
First, the Sam 2000 test was administered prior to the 
paper-based test, thus the students may have done better 
on the paper-based test because they benefited from the 
experience with the Sam 2000 test. Second, the Sam 
2000 test did not award any partial credit. 

FIGURE 7 

COMPARISON TO PAPER-BASED TEST 
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Exam preference. This question evaluated the students’ 
preference for a particular exam medium. Students were 
able to rate their preference for either a paper-based 
exam or a computer-based exam. As shown in Figure 8, 
28% of the students felt neutral, neither preferring the 
SAM exam nor the paper-based exam . Of students who 
showed a preference, 58% (42% of the total respondents) 
preferred the Sam 2000 exam and 42% (30% of the total 
respondents) preferred taking a paper-based exam. 

FIGURE 8 

EXAM PREFERENCE 
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The responses to this question are interesting because on 
average the students scored higher on the paper-based 
exam then on the computer-based exam. Although the 
majority of the studentsscored higheron thepaper-based 
test and felt thatit was easier, as reported in the previous 
question, the students were favorable toward the CBA 
because of the novelty of the testing e nvironm ent. Other 
positive comments about the software included the 
organizational feature of the testing forniat (i.e., at any 
point during the exam, students were able to view 
exactly which questions were completed) and the 
independence amongst questions (i.e., an incorrect 
response to a particular question did not affect the 
response to another question.) Most of the students 
agree with the sentiment — “in a computer class, tests 
should be administered on a computer.” 

Finally students were asked whether or not to consider 
computer-based assessment in the future. 

Future use. Based on the students’ experiences in the 
course, students provided their recommendations on 
using Sam 2000 in future computer classes. Figure 9 
reveals that while 55% ofthe students would recommend 
the continued use of the testing medium, 27% of the 
students felt neutral about the decision and only 19% 
would not recommend using it in the future. 
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FIGURE 9 
FUTURE USE 
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Student Examination Performance 

Two approaches were taken in order to gather data 
concerning student performance. Both tests were scored 
on'a one-hundred point scale. The average score on the 
SAM exam was 84.8%, while the average score on the 
traditional paper-based test was 93.1%. The grade 
distribution is shown in Figure 10. 

FIGURE 10 

COMPARISON OF TEST SCORES 
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Another reason why the scores are higher on the paper- 
based test was that the instructor was able to provide 
partial credit. As mentioned previously, no partial credit 
was awarded on SAM. 

Faculty Perceptions 

From the perspective ofthe course instructors (both full- 
time faculty and adjuncts), there is a general consensus 
that SAM is a success. Instructors found that the 
benefits of using SAM include less time writing exams, 
reduced grading time, simplified record keeping of 
grades through the reporting features, and improved test 
validity through post-test statistical analysis. 
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Figure 1 1 shows the difference in student’s performance 
between the computer-based test and the paper-based 
test. Twenty-one percent of the students did the same or 
better on the SAM exam while the remaining students 
did betteron the paper-based exam. Forty-five percent 
of the students scored higher on the paper-based test by 
one to ten points. The remaining 34% improved their 
scores by more than ten points. 

The improved paper-based testscore canbe attributed to 
the fact that the paper-based test was administered after 
the SAM test. Students were administered that SAM test 
during one class period. At the end of the exam period, 
students were able to review their results. During the 
next class meeting, the students were administered the 
paper-based test. 



Despite the overwhelming benefits of using SAM , there 
are drawbacks. SAM has a limited test bank; although 
multiple tests are available, questions are often re-used 
by instructors from semester to semester. Furthermore, 
instructors cannot add or edit test questions to the test 
bank. The test-bank contains some poorly worded 
questions. There are no constructed response questions 
included on the exam ; each question tests a single skill 
and not a series ofsteps. The most notable drawback of 
SAM is that instructors cannot see every keystroke a 
student performs. This is problem atic when a student is 
marked wrong on a problem and he/she insists that they 
did the problem correctly. There is no way available to 
review the steps they performed. 

Comparing the CBA to the paper-based exam shows that 
there are benefits and drawbacks with each. Instructors 
agree that the most significant benefit of using SAM is 
the ability to evaluate students’ mastery of specific skills 
that can only be assessed by recording interim 
keystrokes. For example, if an instructo r wants to te st a 
student’s mastery of the “Copy and Paste” commands, 
there is no feasible way to evaluate this through a p aper- 
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based exam where the instructor sees only a final 
product. Without watching a student use the copy and 
paste functions, there is no way to determine if the 
student simply retyped the text, rather than pasted it, as 
directed, in a specified location. 

On the other hand, SAM test does not produce a final 
product, just a series of skills. In the “real-world”, 
employees are evaluated on their final product, not 
interim keystrokes. 

CONCLUSION 

This study of using a CBA’s for assessment was based 
on student experiences evaluated with questionnaires and 
informal interviews, reactions of instructors (both full- 
time faculty members and adjuncts), and a comparison 
of Sam 2000 to the traditional testing format as 
described above in terms of measuring student abilities. 

Results from the 1 33 student questionnaires and various 
informal interviews were used to evaluate students’ 
attitudes and perceptions toward computer-based 
assessment. Students evaluated ease of use, fairness of 
grading, perceived impartiality, perceived difficulty, 
perceived level of performance. Results showed that 
students demonstrated an acceptance of the CBA and 
even a recommendation for future use. 

As technology is becoming the norm for delivery of 
courses, computer based assessment will become the 
norm for delivery oftesting. This study investigated the 
use of skills-based tools in the assessment process and 



provided an overview of the techniques, the advantages 

and disadvantages, and an evaluation of the perceptions 

of students and faculty who use them . 
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ABSTRACT 

The authors share their personal experiences and try to give the reader a better understanding of what is involved 
in the application of telematic education and discuss the most important pitfalls and management issues regarding 
telematic education. An interpretive research approach was followed whereby thefeedback of students was evaluated 
regarding their positive and negative experiences in a specific telematic education environment. 



INTRODUCTION AND BACKGROUND 

Telematic education could be regarded as one of the 
most challenging fields for tertiary institutions today. 
Although the technology allows us to basically do what 
we like interms of giving the student service and support 
via the Internet, many questions stay unanswered in 
terms of what is good practice and what is bad practice. 
The authors of this paper share their personal 
experiences and try to give the reader a better 
understanding of what is involved in the application of 
telematic education and discuss the most important 
pitfalls and management issues regarding telematic 
education. The authors assume that the readers are 
familiar with all the concepts and definitions o ftelematic 
education and do not address it, except for giving the 
following citations: 

Telematic education (tele-: over a distance; 
-made: by means of) refers to the way in which 
effective and efficient flexib le teaching/learning 
environments have been created for you. 
Modern communication and information 
technologies(telematics) are skillfully combined 
into flexible educational delivery systems. 



The Rector of the University ofPretoria,Prof. Johan van 
Zyl initiated the development of a formal Technology 
Plan for the University, when he took office in 1997. 
The aim of the plan is to enhance the University’s 
competitivepositionby ensuringthat technology is used 
in the best possible way to support its service delivery 
mechanisms. Based on this plan a telematic projeetteam 
started searching for a web-based software package that 
would satisfy the needs of all the academ ic departments 
at the university of Pretoria in the same year. The 
Department of Telematic Education was laterestablished 
and today serves as a supporting department to assist 
academic departments in establishing their own Web- 
basedtelematic environments for their different modules 
using the web-based package. 

Unofficially the Department of informatics at the 
University of Pretoria had already been involved in 
telematic education since the introduction of their own 
website in 1995. Apart from using their web-site as an 
informationsource or marketing toolfor everyone to see 
what they are offering, web-site pages were created for 
all the under graduate and post graduate courses 
(modules). These pages can only be accessed with a 
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password and are mainly used as a resource for course 
material and general announcements. 

The Department of Informatics established their first 
official telematic web-site for a first year course in 1999 
with the support of the Telematics Department. The 
initial strategy was to migrate all part time modules to 
telematic module offerings, starting with the first-year 
level modules. This case study is based on experience 
and feedback from under graduate students (first year 
part time) as well as postgraduate students (MBA part 
time and modular 1 ). 

RESEARCH APPROACH 

An interpretive research approach was followed to 
gather information about how students experience the 
telematic education environment. Apart from having 
several personal discussions with students, we also put 
a request on the bulletin boards (using the web-based 
package) of both the under graduate and post graduate 
modules to mention three positive and three negative 
aspects of the personal experiences they had with the 
telematic approach of education. The larger amount of 
responses came from the postgraduate web-site. The 
reason for that might be the fact that post graduate 
students were all MBA students and were mainly “off’ 
campus and therefore were more involved in 
experiencing telematic education. Stated differently, 
they were more dependent on the web-site in terms of 
service and support. 

THE TELEMATIC EDUCATION 
SOFTWARE PACKAGE 

The University of Pretoria uses a web-based package for 
telematic education, which offers a large variety of 
facilities to both the student and the lecturer. The 
following is a brief summary of the most important 
facilities offered by the package. 

• Students can register online and pay online. 

• Students can access all course material for a specific 
module. 

• Students can communicate via the bulletin board 
(discussion board) to lecturers and colleagues and 
discuss any module related issues with them. The 
discussion board also allows the users the attach files 
to messages like in the case of normal e-mail. This 
could be done on a public or private basis. 



• All marks deserved by students for a specific model 
can be entered online and are immediately available 
for students to access. 

In order for lecturers to use the package effectively, 
extensive training isneeded. Although lecturers can be 
trained to act only as users and not as designers, it is 
strongly recommended that they should be trained as 
designers. This will put them in a position to manage 
their own courses (modules) completely and cut out 
many frustrations of dependency on other people. 

THE NATURE OF TELEMATIC EDUCATION 
IN THE DEPARTMENT OF INFORMATICS 

The first and maybe one of the most important starting 
points to ensure success in telematic education, is to 
establish sound relationships between the lecturers and 
the students. According to Leonard (1998) relationships 
of such a nature is complex and intrigue. Elements that 
play a very important role in this regard are: trust, 
commitment, co-operation, knowledge, technology and 
support. The process of establishing and maintaining 
sound relationships between students and lecturers in a 
telematic education environment should therefore be 
managed properly, keeping in mind the importance of 
these elements. This is sanctioned by Meyer (2000) who 
states that Telematic Education emphasizes the use of 
technology that will enhance the teaching and learning 
environment and if managed properly, Telematic 
Education has the ability to address the learning needs of 
all learners concerned. 

In general one can say that telematically delivered 
programmes are based on the innovative integration of: 

• contact tuition; 

• paper-based distance education; 

• electronic education through technology-driven 
learning packages that use: 

• interactive m ultimedia; 

• virtual campus technology that integrates with other 
technologies, such as the Internet; 

• and interactive television teaching and video 
conferencing. 

In the case of the Department of Informatics the web- 
based part is mainly used for providing study m aterial to 
students, giving them access to their marks and to 



Proceedings of the 16 h Annual Conference of the International Academy for Information Management 




207 



191 



communicate via the discussion board. Contact sessions 
were minimized to two or three per semester. 

One ofthe immediate assumptions or expectationson the 
users’ (lecturers’) side was that this approach towards 
training could only make life easier. In our case we 
thought that this approach would ensure less work 
because our part time classes were “replaced” by the 
telematic approach. We soon realized that this is not 
really the case. From the students’ viewpoint it is fair to 
say that they new thatthe part time classes were falling 
away, but they expected an even bette r “service” than in 
the past when contact sessions of at least once a week 
were available. 

To overcome the problems that were created by certain 
“false” expectations on both the lecturers side and the 
student side, and the feet that one of the under graduate 
Informatics modules were of a practical nature 
(programming in Natural) a lot of planning was 
necessary. The following is a summary of the most 
important obstaclesand how we triedto overcomethem: 

* Telematic students want “better” service and support 
levels because they feel “neglected.” In order to 
achieve this, we decided to allocate a specific lecturer 
to a telematic module. This lecturer has the 
responsibility of watching the bulletin board every day 
and to react to all messages. The lecturer is also 
responsible for uploading all relevant study material 
onto the “notes page.” 

* In order to ensure that communications between 
students and the department stays on a sound basis, the 
above-mentioned lecturer has the primary 
responsibility of receiving and sending information to 
the students on a daily basis. 

* Because of the easiness of communication (e-mail and 
the discussion board) we sometimes experienced an 
explosion of messages. These messages also came 
from some full time students who discovered the web- 
site. In order to overcome this problem we decided to 
“block” the full time students from getting access to 
the telematic web-based package. This was seen as a 
temporary solution to the problem. We soon learnt 
that students want a quick response to their messages 
(e-mail or bulletinboard) and are ready to complain at 
higher levels if they don’t get feedback soon enough. 



This forced us to communicate with students on a 
daily basis. 

* Messages of an encouraging nature are placed on the 
discussion board as often as twice a week to give the 
students the feeling that the department “looks after 
them 

* Although the amount of contact sessions were much 
less than in the past, these contact sessions are 
arranged to co-inside with academic activities like 
tests. In other words we tried to arrange them in such 
a way that they couldbe used in an optimal way — not 
only for revisiting the academic work but also to write 
a test on previous work. Furthermore, the students get 
the opportunity to portray their experiences and 
feelings regarding the telematic approach. 

* Slides of the lectures need to be more explanatory 
because telematic students don’t have the opportunity 
to listen to the “life” lecture. 

* We overcame the problems we experienced with the 
practical module by allowing students to do their 
practical work at home/word and to submit their 
answers via the web-site. We also introduced a 
practical contact session once per week for those 
students who really need more personal support with 
the programming . 

The service and support components that we argue are 
necessary for a successful Telematic Education 
environment is displayed in Figure 1. 

How Do the Students Feel About the Telematic 
Approach? 

In general the feedback we received was very positive. 
The overall feeling of students about the telematic 
approach could be summarized in the words of three 
students’ feedback: 

“Wonderful product!!” 

“...Awesome!!” 

"... an absolute great system and I am sure 
continual use of it will prove to be very 
successful” 
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FIGURE 1 

THE SERVICE AND SUPPORT COMPONENTS OF TELEMATIC EDUCATION 
IN THE INFORMATICS DEPARTMENT (UP) 




Paper B-oscd 
Dhfta r*e Education 



Mai 



Lecturer 

communication/support 
via e-mail and bulletin 



In the rest of this section some of the most important 

feedback received from students are analyzed briefly. 

Student 1: 

Positive points 

1. Comprehensive; i.e., it contains all the details 
necessary to need and to know about each subject. 

2. The posting of articles and grades. 

3. A continuous link between the lecturer and students. 

Negative points 

1 . It is a lengthy process to get your way around the 
system; i.e., from one subject to another and in a 
specific subject. 

2. I am struggling for a week now to get continuous 
access (day and night) to the virtual campus. 



3. There are some headings in each subject that could 
have been combined into one i.e., discussions, webet 
mail & webet chat. 

Isn’t that part if the same thing viz. electronic 
communication. 

Student 2: 

Positive 

1. Faster feedback on assignments/tests. 

2. Detail on syllabus and calender, makes it easier to 
plan. (Although the IT calender is 2 days out for 
some students.) 

3. Vehicle to post addisional material & info without 
making copies — distribution of large volumes of 
electronic info. 
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Negative 

1. Some lectures do not use WebCT. After 3 months 
there’s not one message on some of the subject 
discussion groups. 

2. WebCT e-mail not in use. Everyone uses their own 
private e-mail. Not even the Director used the 
WebCT e-mail for his last message to MBA 
students. 

3. Time delays in responses from lecturers, to 
questions posted to WebCT. 

Student 3: 

Positive 

1. Can access all relevant information from anywhere 
at any time. 

2. When I have misplaced my study guide, I can 
always have a look on WebCT. 

3. Improve the information exchange between lecturers 
and students (it is often impossible to get hold of 
lecturers personally or telepho nically). 

Negative 

1 . Uncertain ty in the beginning about the features(the 
introductory session helped). 

2. I have not been able to get hold of the ITM Notes (I 
do not know if I am following the wrong links or if 
they have not been posted yet). 

3. Having to type in my password 3x to get to the 
discussion board, is a nuisance. 

Student 4: 

Positive 

1 . Quick flow of information, the discussions and the 
update. 

2. A more open platform than individual e-mails. 

3. Overall scope-academic and non academic 
information. 

Negative 

1. Facilities set to support WebCT disorganized e.g., 
who is in charge atthe lab reserved for this purpose. 
No idea who should provide password to use the 
computers. 



2. It is not always easy to download the attachments 
depending on whether, for example it is in 
PowerPoint, Access, Excel etc. 

Student 5: 

Positive 

1 . Having marks available 24 hours a day. 

2. Quick interaction with and response of lecturers. 

3. Easy reference to study-guide, assesment, mark 
allocations, etc. 

4. Ability to download reference articles straight from 
WebCT. 

Negative 

1. My personal server is not always operational. 

2. Having to re-enter id and password twice. 

Brief Analysis of Student Feedback 

Out of the above student feedback (which were quoted 
precisely as the students sent it viae-mail) it is clearthat 
they enjoy the facility of having online access to all the 
components and facilities offered for a specific course. 
The feedback also emphasizes the fact that they 
appreciate contact with lecturers on a regular basis. 
Furthermore the easy electronic download of study 
material saves time and money for them. 

The most important warnings given by students for the 
success and acceptance of this approach is proper 
training of the users (students and lecturers) and the 
effective management of the web-based package by 
lecturers. They expect from lecturers to communicate 
with them every day and to upload all the important 
study material onto the “notes page” on a regular basis. 
In other words, itis and will only be a pleasure forthem 
if the web-based approach gives them better service and 
support than normal class contact This on the other 
hand asks even more energy from the lecturer! They 
expect from lecturers to practice what they preach in the 
sense that they want lecturersalso to understand and use 
the package on a regular basis. Last but not least, the 
most important element that impact negatively on the 
telematic approach is the availability of the technology. 
If the system is down or if the response time is slow or 
if the general technical support from the university ’sside 
is unsatisfactory, students tend to get negative towards 
this approach. 
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CONCLUSION 

In their search for a (proper) university education, 
students enroll with certain expectations from the 
university, departments and staff members. These 
expectations can be summed up as gaining knowledge 
that is relevant to their working environment and 
personal enrichment, and receiving the necessary service 
and support from lecturers. This paper concentrates on 
the changing approach and attitude towards education 
and learning because of the employment of new 
technology. Telematic education has brought many 
challenges for both student and lecturer. 

Lecturers should always b e aware of the expectations of 
their students. At the same time lecturers should 
comm unicate their expectatio ns to students. Because of 
the limited opportunities for personal contact, serious 
relationships should be built between students .and 
lecturers. These relationships should be built on and 
enforce mutual cooperation trust, commitment, co- 
operation, knowledge and support. Both parties should 
be aware of the necessity to change their respective 
attitudesand approaches towards the learning activity, in 
order to adapt to the changing environment of education. 

Telematic students have the same basic needs and rights 
as full-time students, e.g. motivation, encouragement, 
certainty, and acces s to relevant information (the right to 
be informed). However, lecturers often assume that 
these needs change immediately to self-motivation, self- 
generated learning, etc. when new technology is 
employed. With the implementation of technology 
certain changes has to be made so that students still 
‘feel’ that their basic needs are looked after. Therefore 



this paper proposes continuous communication with 
students in order to understand students’ needs, and 
providing timely information. In a support capacity the 
lecturer should think of ways to complement 
(supplement) the lack of contact. 

The telem atic experience in itself is a learning process. 
Students should gradually (systematically) become 
aware of the larger responsibility that they have towards 
themselves. They should eventually manage their own 
studies, where the lecturer gives less and less support. 
However, this is a timeous process. Students should 
always feel that they get the support needed. 

ENDNOTE 

1. Students that attend classes in a two-week block 
session once or twice a semester. 
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Extended Abstract 



Information technology (IT) has become pervasive in the 
healthcare industry. Many may view the Internet as a 
strategic healthcare tool. It has been suggested that 
Internet-based health applications (IHA), for example, 
electronic medical records, electronic billing, patient 
scheduling, patient monitoring, remote consultations, 
email, etc., can enable the health care industry to reduce 
its inefficiencies and errors in the care delivery processes 
(HIMSS/IBM Leadership Survey, 2000). While the use 
of IT in healthcare has increase tremendously, physicians 
still have not fu lly embraced the valuable resource of the 
Internet. 

We exist in an age where information technology (IT), 
information systems (IS), the Internet, electronic com- 
merce (E-commerce), e-business, telemedicine and even 
e-health are academically recognized, industry pro- 
moted, and governmentally sponsored. Within the 
United States, our daily lives have become increasingly 
computerized and electronically connected. Information 
systems and information technology are prevalent in 
every sector of our society, including healthcare. The 
healthcare industry envisions information technology, 
particularly the Internet playing a pivotal role in the 



future delivery of e-health. Coile, (1999) cites the fol- 
lowing benefits of an Internet-enabled medical practice. 

• Electronic medical records available on a 24-hour 
basis; 

• Continuous clinical data, e.g., hospitalized patients; 

• Disease management programs; 

• Patient monitoring, e.g., chronically ill, home care; 

• Physician communication w ith patient via email; 

• Collegial consultations without geographical boun- 
daries; and 

• Internet-enabled consumer eligibility, claims pro- 
cessing and electronic payment. 

However, before such benefits can be realized, the 
adoption of IHA by key players, particularly physicians 
must occur. Despite the purported advantages of IT 
investments in healthcare many physicians do not widely 
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use Internet-based health applications in their clinical 
practices. Physicians often misunderstand thefunctions 
and full potential of the Internet (Wang & Song, 1997). 
A 1998 survey by Healthcare Financial Management 
reported that forty percent of the p hysicians p oiled stated 
that they would probably not use computers or networks 
for clinical purposes even if training were provided and 
services were made available free or at a very low cost 
(Fotsch, 1998, p. 27). Health & Health Care 201 0 report 
that less than 5% of physicians use computers to record 
all clinical information for an average patient. Clearly 
the recent events concerning terrorist activities in the 
United States have highlighted the already growing 
tensions between the perceived benefits of medical 
informationtechnology (e.g., storageofpatients’ records 
such as DNA for identification purposes) and our civil 



liberties. The perceived threat to personal privacy and 
other leading barriers to the adoption of IHA persist 
(e.g., security/confidentiality concerns; unwillingness to 
share personal/medicalinformation;unwillingness to use 
a credit card on-line; cost of computers and on-line 
access, computer literacy, lack of universal Internet 
access, etc.). 

Grounded in the extended Technology Acceptance 
Model (TAM2) theory (Venkatesh & Davis, 2000), this 
research presents the findings of a field study, which 
examined Hawaii based ped iatricians’ intention to adopt 
Internet-based health applications. This paper is a 
healthcare + Information theory research project in 
progress. 
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ABSTRACT 

MIS Advisory Boards can be a valuable asset in achieving the goals of a university ’s MIS program . A search of the 
literature reveals much indirect evidence of the positive role that advisory boards can play in promoting the interests 
of MIS programs. In this preliminary investigation , we surveyed undergraduate MIS Advisory Boards concerning 
their structure and activity sets. We found that for many MIS programs, advisory boards are an untapped and 
underutilized resource. 



INTRODUCTION 

In 1990, the MIS program area at Indiana State 
University, in conjunction with our Decision Sciences 
program area, formed an advisory bo ard. The purpose of 
the advisory board was to provide a vehicle for 
improving the quality of our program areas and to 
strengthen communication channels between our 
programs and the companies who hire our graduates. 
The advisory board was formed to service our 
undergraduates. 

The purpose of this research is to explore the nature and 
scope of MIS advisory boards across North America. 
We examined the basic structures and the activities of 
MIS advisory boards. This exploratory study lays the 
groundwork for future studiesdesignedto determinebest 
practices and standards which are associated with 
successful undergraduate MIS advisory boards, and 
suggests that most MIS programs are not utilizing this 
potentially valuable asset. 



REVIEW OF LITERATURE 

A review of the literature on MIS advisory boards and 
learning was performed to determine what is formally 
known about the relationship between these boards and 
MIS programs. The search revealed little direct 
knowledge of advisory boards, but the literature hints 
strongly at the potential and utility of those boards. 

At the most obvious level, companies can provide 
valuable insight into the knowledge and skills needed by 
employees and the currency ofthe MIS program. There 
is a long history of surveys of employers to glean these 
insights (Doke and Williams, 1999; Womble, 1994). 
Formation of MIS advisory boards can lend depth to 
these insights. 

Money (1994) has proposed that universities should 
explore new channels of product delivery off-site 
courses, e-education, etc. Money also calls for 
broadening students' educational experience and speaks 
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of the “content-based dangers of IS curriculum because 
of limited linkages between Universities and 
practitioners’ worlds (p . 50), citing the work of Spruell 
and Franz ( 1993). Argyris ( 1 990) theorizes that support 
for the educational experience of the student is needed 
beyond the classroom and program of course work. 
Each of these papers supports the broadening of 
educational experience throughcooperativeexperiences, 
internships, field trips, guest speakers, etc. We see MIS 
advisory boards as playing an important role in 
facilitating each of these activities. 

Cale (1994) provides an example of a very intense 
broadening and deepening linkage between the 
University and practitioner world through student work 
with a mentor company an activity that can be 
facilitated through an MIS advisory board. Fry (1993) 
also makes a case for tight linkage between education 
and industry. This work describes successes from a 
cooperative curriculum development program partnering 
academics and industry representatives. Jacobs (1995) 
discusses the role of an Advisory Board and how it can 
tighten linkages between academics and 
university including the providing of speakers to 
students. 

It is difficult to underestimate the potential of mutually 
beneficial partnerships between Universities and 
industry. Lazer and Norcio (2000) extol the virtues of 
these partnerships for research and for training. We see 
advisory boards as an important first step in the 
development of these partnerships. 

Pierce and Henry (1994) found a difference between 
how educators and IS professionals perceive ethical 
issues, practices, abuse opportunities, extent of abuse, 
etc. One example of these differences is that 
professionals see ethical issues arising more frequently 
than do academics (i.e., more of a real world problem). 
Because of these differences, educators may not be 
teaching ethics as knowledgeably as it could be taught. 
The authors conclude that, since teaching ethics is not 
easy (especially given that academics and IS 
practitioners have different perceptions), it is important 
that educators and professionals form a partnership. 
They further conclude that it is important to instill 
ethical standards in IS students. MIS advisory boards 
provide an opportunity to foster this partnership. 

Dick, et al. (1995) suggest colloquium evenings and 
short seminars as appropriate vehicles to deliver an 
understanding of professional ethical expectations, the 



role of IS professionals, negotiation skills, and where an 
IS career might lead. Benbasat etal. (1980) have noted 
that too much emphasis has traditionally been placed on 
developing students’ technical skills and not enough on 
behavioral or organization skills. The aforementioned 
colloquia and short seminars are one means of increasing 
behavioral and organizational skills. MIS advisory 
boards can serve as a platform for reinforcing students’ 
understanding of these critical skills. 

One informal method of workplace learning is dialogic 
learning learning through informal discussions with 
other workers. Gardiner and Singh (91) have found that 
cooperativeeducationprograms in Science and IS result 
in students whose transition to the workplace is less 
difficult than their (non-coop) peers. Cooperative edu- 
cation is a strategy of applied learning which is 
structural, developed, and supervised by a university in 
collaboration with one or more employing organizations. 
The philosophy underlying cooperative education is 
closely aligned with the experiential learning strategy 
(Galliers, 1987; Lee 1987; Little and Magetson, 1989; 
and Wersky, 1989). Small, etal. (1998) have found that 
cooperatives can provide another avenue for students to 
engage in experiential learning, including through 
community service projects. As with previously 
mentioned activities, MIS advisory boards can augment 
cooperative efforts. 

Romm and Wong (1997) have found that the best 
method to teach s oft skills is through case study. Cragg 
(1998) found that site visits serve as a form of live case 
study for IS courses. The site visit brings reality to the 
students and provides variety in the learning experience 
while encouraging action learning. Further, the author 
contends that a site visit allows for a rich opportunity for 
students to explore theory and practice, expose students 
with little experience to the complexities of 
organizations, and help educators to keep up to date w ith 
current practices and problems. MIS advisory boards 
can play a useful role in facilitating site visits. 

Recently Duncan (2000) provides perhaps the most 
direct evidence ofthe value of advisory boards. Duncan 
describes nine different methods for forming 
partnerships between universities and industry. She 
argues that advisory committees, composed of alumni, 
MIS managers, and recruiters, are an excellent means of 
determining which hardware, software, and applications 
should be adopted by the university. Duncan argues that 
the advisory committee is in a good position to provide 
input regardingthe relevancy of the content ofindividual 
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courses and the curriculum as a whole. She advocates 
that the committee meet as a body at least once a year, 
with an agenda including a report on hardware and 
software in use by the university, curriculum changes 
being considered, and any problems since the last 
meeting of the committee. She notes that faculty 
requests for resources and curriculum changes that have 
been supported by an advisory committee are usually 
looked upon more favorably by the campus 
administration. She further notes, based on personal 
experience, that advisory comm ittee memb ers are more 
likely to enter into other form of “partnership with the 
university. 

Summary of the Literature 

We found a substantial body of literature indicating that 
MIS advisory boards can facilitate a broad set of 
pedagogically valuable activities. Activities such as 
seminars, colloquia, speaker visits, visits to industry 
sites, and promotion of professional ethical standards 
enrich and broaden the student learning experience. In 
addition, advisory boards strengthen ties between 
universities and industry, and provide valuable input on 
hardware, software, and the curriculum. This study 
attempts to determine the scope and frequency ofsome 
of these activities in today's MIS advisory boards. 

METHOD 

This study was carried outin phases. First, the literature 
on advisory boards and MIS programs was reviewed to 
determine what is formally known about the re lationship 
between advisory boards and student learning. Second, 
a search of the Internet was conducted in an attempt to 
find a representative sample of convenience of MIS 
advisory boards (Appendix A). Thirty-nine organi- 
zations were identified and included in the study. 

In the third phaseof the study, representatives from these 
advisory boards were contacted via email and asked to 
respond to a one-page survey (Appendix B). We sent 
out three waves of e-mails at approximately 10-day 
intervals. We received responses from only thirteen of 
the organizations, or 33 percent. 

Surprised by the response to our targeted list of 39 
schools, we decided to supplement the initial survey. 
We sent surveys to anew set of schools drawn from a set 
of participants at the 2000 Conference of Information 



Resources Management Association. We sent surveys to 
thirteen of these schools and received a response from 
four, or 31 percent. 

RESULTS 

We are disappointed in the low response rate to our 
surveys. If we were disappointed with the low response 
rate, we were astonished that our Internet search proved 
to be so unreliable in locating respondents who actually 
had the kind of undergraduate MIS advisory board we 
are interested in. We found many other kinds of 
advisory boards, most of which fell in these four 
categories. 

1. IT advisory board for University serving as a 
board of directors to formulate policy for the 
university’s IT department (as opposed to the MIS 
program). 

2. IT advisory board for Schools of Business serving 
in the same capacity at number one above. 

3. An advisory board for Schools of Business serving 
as curriculum, etc., advisors for the entire School of 
Business. 

4. An MIS advisory board serving in an advisory 
capacity for the graduate MIS program. 

Of the thirteen respondents, only five have MIS advisory 
boards serving the undergraduate program . Although it 
is somewhat presumptuous to attempt to characterize 
undergraduate MIS advisory boards base on only five 
responses, we summarize the results so far in Table 1. 

Based on the limited responses above, it is apparent that 
advisory boards are providing basic services, but are not 
being fully utilized for financial support. 

In addition to the above results, the five respondents 
have an average of 17.4 members sitting on the advisory 
board, and the boards meet an average of 2.1 times per 
year. Fortwo ofthe schools, we found that members run 
the gamut from CIO to entry-level positions. For the 
otherthree schools, members were CIOs and other high- 
ranking executives. At Indiana State University, not 
included in the survey, the question ofwho serves on the 
advisory board is a dilemma. High-level executives are 
in abetter position to fund requests for resources. Lower 
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TABLE 1 

SURVEY RESPONSE DATA 



Question 


Number of “yes” 
responses 


Percent of “yes” 
responses 


Do you allow more than one advisory board member from the same 
company? 


4 


80 


Do the majority of the companies regularly hire your M IS 
graduates? 


4 


80 


Do the advisory board members provide money for awards and 
scholarships to MIS students? 


3 


60 


Do the advisory board members regularly provide feedback on the 
MIS curriculum on at least an annual basis? 


5 


100 


Are internships offered by the majority of the MIS advisory board 
companies? 


3 


60 



\ 



level members tend to exercise direct control over the 
hiring of our graduates. This study shed no light on 
resolving the dilemma. 

Of the eight respondents who currently did not have an 
advisory board, two (25 percent) indicatedthat planning 
for an advisory board for the near future was in place. 
Although we can draw no firm conclusions based on 
such slender response, we can state that there is 
significant interest in establishing advisory boards. 

We received four responses to the second set of 
questionnaires. None of these four respondents had an 
MIS Advisory Board. This result suggests that there are 
many schools which are not taking advantage of the 
potential that an advisory board provides. 

( 

IMPLICATIONS 

This research provides evidence that advisory bo ards are 
very much an unexploited resource for MIS programs. 
Several of the respondents who. do not currently have 
undergraduate MIS advisory boards have indicated that 
advisory boards are a good idea, and some have plansto 
initiate such boards in the near future. Our research 
indicates that advisory boards can play a large role in 
assisting MIS programs than what might at first be 
envisaged. Finally, nearly all our respondents indicate 
an interest in receiving the results ofour study, evidence 
that this under-researched aspect ofMIS education is of 
interest. 



More research on advisory boards is needed. Our low 
response rate is evidence that MIS advisory boards are 
much more uncommon than we had believed. We need 
to confirm whether or not this apparent lack of MIS 
advisory boards is real. We remain convinced that 
advisory boards can be a valuable asset to an MIS 
program, and we hope to persuade MIS programs that 
the adoption of an advisory board canprovide a real and 
tangible service to their students. 
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APPENDIX A 



1. http://www.slis.indiana.edu/Degrees/mis/misab.html - IU Bloomington School of (MIS program — master of 
information science). Chairperson = Rob Kling, kling@indiana.edu, (812) 855-9763, fax (812) 855-6166 

2. http://www.sba.udayton.edu/mis/adv_board.htm U of Dayton MIS and Decision Sciences Dept, school of 
Business Administration Chair = Jeffrey Hoffer (professor and chair), hoffer@udayton.edu, (937) 229-2938 

3. http://rlieb.ba.ttu.edu/misAdvBoard.html TX tech MIS Advisory board (under construction) 
http://www.ba.ttu.edu/rlieb/rn isadv.htm TX Tech U MIS Advisory Board Information Systemsand Quantitativ e 
Sciences, School of Business Administration. Contact person = Robert Lieb, robert.lieb@ttu.edu 

4. http://www.umsl.edu/divisions/business/mis/mentor/advisory_board.html U of MO at St. Louis 

http://www.umsl.edu/divisions/business/mis/board_visitors.html U of MO at St. Louis, Management 
Information System, College of Business Administration, Contact person = Dr. Douglas Durand, Dean, at 
Durand@umsl.edu, or Dr. Robert Nauss, AS Area Coordinator, atRobert_nauss@umsl.edu 

5. tp://www .unex.ucla.edu/shortcourses/directory/it_advisory.htm - UCLA (IT Advisory Board) Computer Science 
Dept, School of Engineering and Applied Science IT Advisory Board member contact person = Dr. Alfonso F. 
Cardenas, cardenas@cs.ucla.edu, (3 10) 825-7550 Computer Science Dept Alumni Advisory Board chair = Dr. 
Don Calhoun, calhoun@standfordalumni.org, (310) 792-9089 

6. tp://ww w.cs. panam. edu/chen/pubin fo/adviso ly/meet ingFOO.html -U ofTX Pan Am (Com puter Science Advisory 
Board). Department of Computer Science, Computer Science Advisory Board Chair = Ruben Galvan (no contact 
info), Computer Science Advisory Board Executive secretary, and Dept, of Computer Science Chair = Dr. Pearl 
Brazier, brazier@panam.edu, (956) 381-2320, 381-3455 

7. tp://www .uic.edu/cba/crim/M inAgnd a/BrdM nJun.htm - U of IL at Chicago (Center for Research in Information 
Management Advisory Board), College of Business. Contact persons = Bob Abrams, rabrams@uic.edu, (312) 
996-2676, Herb Zuegel, hzuegel@uic.edu, (312) 996-2679 

8. tp://www .slu.edu/colleges/business/dsm is/oscmboard.htm - St Louis U (advisory board of the OSCM 
concentration: Operation and Supply Chain Management) Department of Decision Sciences & MIS, School of 
Business & Administration Contact person = Kevin Forness, FornessK@slu.edu, ordsmis@slu.edu, (314) 977- 
3878 

9. tp://cobweb.creighto n.edu/m is/advisory Board.htm - Creighton U, IS Dept’s advisory board, Department of 
Information Systems, college of Business Administration, Contact person = Dr. Michael Echols, Executive 
Director of Creighton Institute for Information Technology and Management, mechols@creighton.edu 

10. p://www .bsbpa.um kc.edu/m ismba.htm -U of MO at Kansas City Management of Information System, School 
of Business and Public Administration Contact persons: 

Nancy Day, Director of Business Administration 816-235-2333 (can’t access e-mail addresses) 

Sidne Ward, Assistant Professor of Management Information Systems, wards@umkc.edu, 816-235-2321 

Roger Pick, Associate Professor ofManagement Information Systems, pick@acm.org, 816-235-2336 

1 1. http://sbaeweb.fullerton.edu/msis/spring%20news.htm - Cal State Fullerton Management Science/Information 
Systems Department, School of Business Administration & Economics Contact person = Dep artment Chair Barry 
Alan Pasternack, Bpasternack@fullerton.edu, (714) 278-2221 
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12. http://wbdc.hcob.wmich.edu/Advisory%20Board.htm - Haworth College of Business Advisory board steering 
committee contact person = Jan Andersen, PresidentJanbeyond@earthlink.net 

13 . h ttp://www. cob. fsu. edu/grad/mba/ad visory.html - FL State U,MBA Advisory Board College of Business Contact 
person = Dr. Joey George, dept of information and management science atjgeoige@cob.fsu.edu, or Christopher 
Campbell, President ofthe MBA Advisory Board, at ccampbell@campbellgroup-i.com, College of Business’s 
phone # (850) 644-6458 

14. http://cba.ksu.edu/org/mis/club/sponsors/Friendbg.htm - Kansas State U Management Department, College of 
Business Administration, Contact persons = Dr. RogerMcHaney, faculty advisor, mchaney@ksu.edu, (785)532- 
7479 Or mis_club@ksu.edu 

1 5. http://www2.uta.edu/infosys/IT AdvisoryCouncil/ - U of TX at Arlington, ISMS advisory board Department of 
Information System sand Management Sciences, College of Business Administration Contact person = Dr. R. C. 
Baker, Dept Chairman, rcbaker@uta.edu, (817) 272-3547 

16. http://www.haas.be rkeley.edu/~citm/advisory.htm 1 - UC Berkeley, The Fisher Center for Management and 
Information Technology Hass School of Business Contact person = Dr. Arie Segev, segev@haas.berkeley.edu, 
(510) 642-4731 or citm@haas.berkeley.edu 

17. http://www.outreach.washington.edu/extinfo/certprog/mri/mri_brd.asp -U ofWA Electronic Information and 
Records Management (Certificate Program) Contact person = uweo@u. washington.edu (206) 543-2320, (800) 
543-2320 

1 8. http://depts.washington.edu/ebiz/program/board.html - U of WA E-Business Advisory Board, business school 
Contactperson = Dr. William E. Burrows, foundationof e-business, burrows@u. washington.edu, (206) 543-4474, 
or Ebizinfo@u. washington.edu 

19. http://www.pontikes.siu.edu/advboard.htm - Southern ILU atCarbondale (Pontikes Center for Management of 
Information’s advisory board) College of Business and Administration Contact person = Dr. Gregory White, 
Director, jbahoric@cba.siu.edu, (618) 453-7791 

20. http://www.sjsu.edu/senate/s82-10.htm - San Jose State U, Information Systems and Computing advisory board 
Information System dept. Contact person = IS Interim Director sksh@email.sjsu.edu, Special Projects 
dling@email.sjsu.edu, (408) 924-7305 

21. http://www.terry.uga.edu/mis/Advisoryboard.htm - U of GA Dept of Management Information Systems, The 
Terry College of business Contact person = Dr. Patrick McKeown, head of the MIS dept, 
pmckeown@terry.uga.edu, (706) 542-3341 

22. http://www2.uta.edu/infosys/ITAdvisoryCouncil/ U of TX @ Arlington Dept ofIS and MS Contact Dr. R. C. 
Baker, dept, chair, at rcbaker@uta.edu. 

23. http://www.uark.edu/depts/cisqinfo/advisory.htmlUofAR Dept of Computer Info Sys and Quantitative Analysis 
Contact Dr. Fred Davis, dept chair, at fddavis@comp.uark.edu 

24. http://www.csun.edu/acct/advcncl.html Cal State Northridge Dept of Accounting and MIS Contact Dr. Earl 
Weiss, dept chair, at earl.weiss@csun.edu or acctmis@csun.edu 

25. http://www.bus.orst.edu/About_the_College/Information_Management.htm OR State U Dept of Accounting, 
Finance, and Information Management advisory council Contact Dr.Ilene Kleinsorge, faculty, at 
ilene@bus.orst.edu 
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26. http://cobe.boisestate.edu/govern/exec/notes.htm Boise State U College Of Business and Economy Advisory 
Council (Computer Information System) Contact Dr. Davis Groebner, dept chair, at dgroebne@boisestate.edu 

27. http://www.bu.edu/ctFpeople/ITboard.html BostonU InformationTechnology advisory board ContactDr. Azer 
Bestavros, Computer Science dept chair, at best@bu.edu 

28. http://www2.clarku.edU/newsite/gsom/connect/index.shtml#advisory Clark U, Worcester, MA, Grad School 
of Management advisory council ContactDr. Edward Ottensmeyer, dean, at eottensmeyer@clarku.edu 

29. http://www. esc. calpoly.edu/department/iac/by laws.html Cal Poly Dept of Computer Science Contact Dr. Sigurd 
Meldal, dept chair, at smeldal@calpoly.edu 

30. http://www.sas.muohio.edu/advisorycouncil/- MiamiU, OH School of Engineering and Applied Science advisory 
board Contact Dr. Marek Dollar, dean, at dollarm@muohio.edu 

3 1 . http://134.53 .40. l/dms/duricydd/mis381 /advisoryc ouncil.htm MiamiU, OH Dept of Decision Science & MIS 
business advisory council Contact Dr. David Yen, dept chair, at yenbc@muohio.edu 

32. http://grad.cgu.edu/~icouncil/Council/Council-Advisory.htm Claremont Graduate University School of 
Information Science, IS Student Council advisory board Contact Joe Vatanasombut, president of the student 
council (student), at vatanasb@cgu.edu 

33. http://home.uleth.ca/man/board/board.shtml U of Lethbridge Faculty ofManagement advisory council. Contact 
Dr. Ali Dastmalchian, dean, at dastmal@uleth.ca 

34. http://www.fredonia.edu/business/baaac/ - State U of NY @ Fredonia Dept of BA and Accounting advisory 
council Contact Patrick McNamara, advisory council secretary, at pmcnamaral 986@kellogg.nwu.edu 

35. http://www.som.umass.edu/som/dev/bac.html U of Mass @ Amherst Business advisory council Contact Dr. 
James Smith, chair of Dept of Accounting and Information System, atjfs@som.umass.edu 

36. http://www.ecsu.ctstateu.edu/depts/edu/ - Eastern CT State U Center for Instructional Technology (CIT) meeting 
Computer Information System Advisory Council (CISAC) 

38. Education Dept, http://www.ecsu.ctstateu.edu/depts/edu/ - Eastern CT State U Center for Instructional 
Technology (CIT) meeting Computer Information System Advisory Council (CISAC) Educatio n Dept. Contact 
Dr. David Stoloff, education Dept, chair, at stoloffd@easternct.edu 

39. http://www.tamu.edu/president/plan/Business.htm (this site mention the informationsystem advisory board) - TX 
A&M University Center for the Management Information System Advisory Board (Home - 
http://cmis.tamu.edu/) Department of Information & Operations Management Contact Dr. George Fowler, 
Director, at gfowler@CGSB.tamu.edu 
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APPENDIX B 

MIS ADVISORY BOARD SURVEY 



Dear MIS Professional, 

Like you, we are involved in MIS and academics. Approximately ten years ago we developed an Advisory Board 
consisting of members from industry to help guide our MIS major. We have asked this Board for advice regarding 
curriculum and also have asked for help for scholarships. 

Now, we would like to learn more about whatother schools are doing with advisory boards. An Internet based search 
leads us to believe your institution m ay also be doing som ething similar. We ask if you would p rovide us w ith a little 
information regarding your undertaking . We have structured a set of questions below and request that you please take 
a few minutes to answer them. A simple yes or no will answer most of the questions. 

After we compile the questionnaires, we will send you a listing of schools participating and a summary of results. This 
information may help all of us working with advisory boards in the future. At a minimum, we will become aware of 
other institutions that we might possibly communicate with in the future to share ideas about benefits of advisory 
boards. 

Please fill in your answer after each question and just use a simple reply to this e-mail. 

1. Do you have an MIS undergraduate major? 

2. a) Do you have a corporate MIS advisory board providing advices and/or resources for MIS major? 

b) If you answer no to the above, do you plan on implementing a corporate advisory board in the near future? 

If you answer no to 2b), you have completed the survey. Thank you for your time. 

3. About how many companies are represented on the advisory board? 

4. Do you allow more than one advisory board member from the same company? 

5. Do the majority of the companies regularly hire your MIS graduates? 

6. How m any time s a year doe s the adviso ry board m eet? 

7. Do the advisory board members provide money for awards and scholarships to MIS students? 

8. Do the advisory board members regularly provide feedback on the MIS curriculum on at least an annual basis? 

9. Are internships offered by the majority of the MIS advisory board companies? 

10. Provide a few small examples of the type of positions currently held by your advisory board members; for 
example, program analyst, project manager, Chief Inform ation Officer. 
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ANALYZING THE FACTORS THAT AFFECT 
INTENTION TO ENTER A POST BACCA- 
LAUREATE BUSINESS EDUCATION PROGRAM 



Richard V. McCarthy 
Quinnipiac University 

George Claffey 

Central Connecticut State University 

Jay E. Aronson 
The University of Georgia 



ABSTRACT 

Post baccalaureate business education programs are organized and administered in a variety of ways by colleges 
and universities throughout the United States. One principle that they have in common is that to be successful, there 
must be a well-integrated curriculum of education and business courses that focus and train the students to become 
the business educators of tomorrow. Prospective students make decisions to enter a post baccalaureate business 
education program based upon the curriculum requirements; however this is not the only factor that influences their 
decision. 

There are many factors that influence a student's decision to enroll in and then subsequently complete a business 
education program. Some of these factors include: the location of the university, the entrance requirements of the 
university, the information satisfaction as perceived by the perspective student when they investigate a university 
program, the cost of the program and the number of credit hours required for completion. This paper studies the 
factors that affect a student's decision to enter a business education program. 



REVIEW OF THE LITERATURE 

Bronner (2000) found that the problems with business 
education programs as ranked by the presidents of State 
of Connecticut colleges and universities with business 
education programs include: 

1 . Lack of funding 

2. Lack of adequate facilities 

3. Lack of support from the school administration 

4. Lack of positive attitude of counselors 

5. Lack of time for professional development 



Anderson and Sinha (2000) concluded that opportunities 
for business education teachers remain limited. 
Additionally, they found that the field is dominated by 
females and that satisfaction was a significant variable in 
choosing to enter the field. 

These issues helped to explain the simultaneous decline in 
enrollment and growth in the number of business 
educators leaving the field. Of particular concern lately 
has been the aging teacher population and shortage of 
business educators. Okula (1999) found that as many as 
fifty percent of business teachers are nearing retirement 
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age. She also noted that over the prior ten years, the 
number of college programs that prepare high school 
business teachers has dropped significantly. Equally 
important, Okula indicates that teacher certification 
requirements in some states make it very difficult for 
business professionals to enter the field. 

LaBonty (1999) provides further evidence of the demand 
for business educators and the issues surrounding 
enrollment in the 1997-1998 NABTE survey. The 
findings indicate that the change in the number of 
enrollments in certification-only students dropped from 
788 certification-only students in 1996 to 475 
certification-only students in 1998, a decline of 40%. 
Additionally, the number of student teachers fell from 
1,529 in 1991-1992 to 752 in 1997-1998. 

The Connecticut Legislature created an 18 member 
Commission on the Teacher and School Administrator 
Shortage and Minority Recruitment (Lohman, 2000). 
The Commission reviewed what other states have done as 
well as what Connecticut accomplished in 1 986 under the 
Education Enhancement Act which raised minimum 
starting salaries. Drawing from other states, the 
Commission considered raising starting salaries (even 
though Connecticut is reported to be the second highest in 
the USA), signing bonuses, loan forgiveness programs, 
assistance with real estate costs, incentives for veteran 
teachers, and incentives for retirees. 

McEwen and King (1998) concluded that keyboarding 
continues to be the most frequently taught business 
subject. They also found that the student teaching 
experience played a significant role in the perception of 
the student’s view of a business education program. 
Jennings (2001) examined national trends in keyboarding 
and found that although it is now taught early in the 
school system, it is considered to be an important skill. 
This finding was also supported by Womble, Adams and 
Stite-Gohdes (2000) who concluded that keyboarding and 
basic computing skills were an essential part of a 
successful business education program. 

RESEARCH QUESTIONS 

The research questions were developed as a direct result 
of observance of prospective students questions while 
they were inquiring about the business education 
program. The following research questions are the focal 
point of this investigation. 

208 



1 . What program requirement factors affect a student’s 
decision to enter a business education program? 

2. What program structure factors affect a student’s 
decision to enter a business education program? 

The purpose of the research is to gain insight into the 
factors that the university should focus on in order to 
improve its ability to attract students into the business 
education program. Figure 1 represents the research 
model that served as the basis for this investigation. 

FIGURE 1 

INTENTION TO ENROLL IN BUSINESS 
EDUCATION RESEARCH MODEL 




METHODOLOGY 

A survey was completed of prospective students, who 
considered entering the post baccalaureate business 
education program at Central Connecticut State 
University, New Britain, CT. Questions were developed 
based upon factors that relate to the program of study as 
well as the university. The factors were drawn from 
observation by the business education chairman at 
Central Connecticut State University, and grounded in the 
literature from prior studies of factors that influence 
student’s decision to enter a program. 

The requirements for entrance into the post-baccalaureate 
business education certification program include an 
undergraduate degree in business or courses equivalent to 
the business core, a grade point average of at least 2.70, 
and passing the Praxis I (a standardized test for assessing 
reading, writing, and mathematical skills required by the 
State). Additional requirements include passing a 
keyboarding proficiency exam (42 words a minute after 
error penalties for 3 minutes with three or fewer errors, 
without correction) and passing a word processing exam 
or equivalent one-credit course. The required Business 
Education certification courses include two methods 
courses (one for skill and one for non-skill subjects), a 
software course, and an organization and administration 
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course. Ultimately students must be admitted to the 
department (professional admission) and produce 
evidence of 500 hours of office experience within the past 
five years as well as write an essay and undergo a 
personal interview. Students must also meet the 
requirements from the School of Education and 
Professional Studies that include courses in foundations 
of education, principles and evaluation. It is estimated 
that most business graduates seeking business 
certification take approximately 39 semester hours of 
course work including student teaching. 

Course scheduling permits full-time students to complete 
the requirements in a year plus a partial semester for 
student teaching. Part-time students generally take two 
and a half years. Connecticut requires a student to pass 
the Praxis II in business education for certification. A 
30-hour master’s degree in Business Education is 
available for certified business teachers. Individuals can 
count 12 credit hours from the certification program 
towards the master’s degree requirements, but all 
coursework must be completed within six years. 

The survey consisted of seventeen questions that utilized 
a five point Lickert scale. Two of the questions pertained 
to information satisfaction. The possible responses 
ranged from strongly disagree (5) to strongly agree (1). 
The factors that were tested are listed in Table 1. 

The sample population was prospective students that 
inquired about the business education program at Central 
Connecticut State University. Central Connecticut State 
University is the only publicly supported university in the 
State of Connecticut that offers a post baccalaureate 
business education program. Each of the factors is 
classified as pertaining to either the program requirements 
or the program structure. Multivariate statistical 

techniques were utilized to analyze the results of the 
survey. Cronbach’s Alpha was used to determine the 
reliability of the questions that comprise the factors that 
were tested. Inferential statistics consisted of Pearson’s 
product moment test for correlation, and analysis of 
variance (ANOVA). 

ADMINISTRATION OF THE SURVEY 

The survey was sent to eighty-six prospective students 
who had requested information on the business education 
program at Central Connecticut State 
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TABLE 1 

FACTORS THAT AFFECT 
STUDENT’S DECISION TO ENTER 
BUSINESS EDUCATION PROGRAMS 



1 


Enrolled in a different degree 


2 


Enrolled in another certification program 


3 


Enrolled in another university 


4 


Number of credits required 


5 


Cost 


6 


Poor job prospects 


7 


Low potential salary 


8 


Inconvenient class times 


9 


Student teaching requirement 


10 


Inadequate facilities 


11 


Keyboarding requirement 


12 


Distance 


13 


Entrance requirements 


14 


Masters degree requirement 


15 


Information satisfaction 


16 


Ease of use 



University. A cover letter from the business education 
chair accompanied the survey requesting that the survey 
be completed and returned in an enclosed pre-addressed 
stamped envelope. The population consisted of one and a 
half year’s worth of prospective students. These 
consisted of individuals that had contacted the university 
requesting program information between August 1998 
and June 2000. Individuals that did not return the survey 
within a three-week time frame were sent a follow up 
request to return the survey. A total of forty surveys were 
returned. This represents a response rate of 46. 5%. The 
very high response rate can be attributed to a sample 
population that was interested in the topic and was 
familiar with the requestor. The respondents 
communicated with him to gather information about the 
business education program, as he is the only advisor in 
the program at Central Connecticut State University. 

There were no incomplete surveys; each of the forty 
surveys returned was usable. Twenty-five of 'the 
respondents were female, and fifteen were male. The 
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returned surveys were encoded into the SPSS statistical 
package, with each question representing a construct. 

The responses to each of the questions ranged from strong 
agree to strongly disagree. Table 2 presents a summary of 
the responses received. 

ANALYSIS OF THE SURVEY 

The three questions that pertain to enrollment in other 
programs were used to analyze the intention to enroll 
construct. The remaining questions pertain to either 
program requirements or program structure. Table 3 
provides an identification of the program requirements/ 
structure grouping. 



A Cronbach’s Alpha test of reliability was performed for the 
constructs that comprise the program requirements and 
program structure. The Cronbach’s Alpha for the program 
requirements was .724 and the Cronbach’s Alpha for the 
program structure was .71; which was suffi-cient to 
conclude the internal reliability of the constructs. The 
constructs that comprised the decision to enroll exhibited a 
Cronbach’s Alpha of .8379. A Cronbach’s Alpha greater 
than .7 is generally accepted as exhibiting a strong internal 
reliability. A Pearson’s product moment correlation test was 
performed for each of the variables as they affect the 
decision to enroll. An Alpha level of .05 was utilized. The 
results are presented in Table 4. Salary and cost exhibited 
a negative correlation (-.249 and -.013 respectively) and 
were dropped from further consideration. 



TABLE 2 

SUMMARY OF RESPONSES RECEIVED 





Strongly 


Agree 


Not a 


Disagree 


Strongly 


I have enrolled in different degree program instead 


Agree 

15.0% 


0.0% 


Factor 

45.0% 


12.5% 


Disagree 

27.5% 


I have enrolled in another certification program 


12.5% 


0.0% 


47.5% 


10.0% 


30.0% 


I have enrolled in a business education certification program at another 


10.0% 


0.0% 


50.0% 


10.0% 


30.0% 


university 

The number of credits (time commitment) seemed too long 


17.5% 


40.0% 


25.0% 


10.0% 


7.5% 


The total dollar cost of courses seemed too much 


12.5% 


17.5% 


50.0% 


15.0% 


5.0% 


The job prospects were too unpredictable 


12.5% 


12.5% 


40.0% 


25.0% 


10.0% 


The salary of beginning teachers is too low 


32.5% 


15.0% 


42.5% 


7.5% 


2.5% 


Classes are offered at inconvenient times 


2.5% 


10.0% 


50.0% 


27.5% 


10.0% 


The student teaching requirement is a problem 


22.5% 


15.0% 


40.0% 


17.5% 


5.0% 


CCSU staff or facilities did not seem inviting 


7.5% 


2.5% 


35.0% 


27.5% 


27.5% 


The keyboarding requirement is unrealistic 


25.0% 


25.0% 


25.0% 


15.0% 


10.0% 


Travel distance to CCSU is too great 


7.5% 


7.5% 


55.0% 


10.0% 


20.0% 


My grade point average prevented me from meeting entrance 


0.0% 


10.0% 


40.0% 


20.0% 


30.0% 


requirements 

I was only interested in finding more information and am satisfied 


7.5% 


30.0% 


30.0% 


22.5% 


10.0% 


The requirement for a master’s degree above certification is too much 


7.5% 


25.0% 


32.5% 


22.5% 


12.5% 


I did not get enough information to make a decision 


2.5% 


15.0% 


32.5% 


20.0% 


30.0% 


The entire process is too confusing 


5.0% 


32.5% 


25.0% 


15.0% 


22.5% 
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TABLE 3 

PROGRAM REQUIREMENTS AND STRUCTURE 



1 


Number of Credits Required 


Requirements 


2 


Cost 


Structure 


33 


Poor Job Prospects 


Structure 


4 


Low Potential Salary 


Structure 


5 


Inconvenient Class Times 


Structure 


6 


Student Teaching Requirement 


Requirements 


7 


Inadequate Facilities 


Structure 


8 


Keyboarding Requirement 


Requirements 


9 


Distance 


Structure 


10 


Entrance Requirements 


Requirements 


11 


Masters Degree Requirement 


Requirements 


12 


Information Satisfaction 


Structure 


13 


Ease of Use 


Structure 



TABLE 4 

PEARSON’S PRODUCT MOMENT CORRELATION 



Variable 


Correlation 

Coefficient 


Time Commitment 


0.031 


Cost 


-0.013 


Job Prospects 


0.109 


Salary 


-0.249 


Class Time 


0.277 


Student Teaching 


0.101 


Facilities 


0.098 


Keyboard Requirement 


0.329 


Travel Requirements 


0.261 


GPA 


0.235 


Information Satisfaction 


0.143 


Master’s Degree Requirement 


0.048 


Ease of Use 


0.097 



Two hypotheses were tested to address the research 
questions. 

Hj : The decision to enroll in a post baccalaureate 
business education program is positively affected by 
the program structure. 

H 2 : The decision to enroll in a post baccalaureate 
business education program is positively affected by 
the program requirements. 

An ANOVA test was performed on the remaining 
variables that comprise program structure and program 



requirements at an Alpha level of .05. The calculated F 
value for the program requirements (3.21) and program 
structure (2.43) exceeded the F value, at six degrees of 
freedom. After we established that each of the null 
hypotheses failed to be rejected, a factor analysis was 
performed to determine the strength of the relationships 
of each of the variables that were tested. 

DISCUSSION 

The survey results demonstrated that there was a positive 
correlation between a student’s intention to enroll and the 
program structure and program requirements. The 
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program requirements provided stronger explanation of 
the intention to enroll. The factors that comprised the 
program requirements consisted of the student teacher 
requirement, keyboarding requirement, entrance 
requirements, master’s degree requirement, and the 
number of program credits. Our research showed that the 
student teaching requirements and keyboarding 
requirement had the greatest affect of the program 
requirements, containing a correlation coefficient of .381 
and .353 respectively. Although it was initially thought 
that entrance requirements would have a strong influence 
on a student’s decision to enroll, the actual results yielded 
a very low correlation coefficient (.043). Our research 
showed that the areas that have the greatest impact on a 
student’s decision to enroll consist of a combination of 
the program structure and program requirements 
variables. 

One of the outcomes of this research has been a change in 
the admission process at Central Connecticut State 
University. Students are now required to complete 
keyboarding and word processing preliminary proficiency 
exams and the Praxis I exam prior to beginning course 
work in the School of Education. 

Our survey was limited by the relatively small, 
homogeneous group. Each of the survey participants had 
prior experience with Central Connecticut State 
University. The prior experience could help to explain 
why cost was not considered as a factor. It would be 
generally recognized by the respondents that Central 
Connecticut State University is one of the lowest cost 
universities in Connecticut. 

FUTURE WORK 

An extension of this study will include an analysis of the 
effect of the structure of the curriculum of business 
education programs. This will include a comparison of 
the strength of the relationship between the business 
curriculum and the student’s intention to enter a business 
education program, as well as the strength of the 
relationship between the education curriculum and the 
student’s intention to enter a business education program. 
To perform this study, the survey will be extended to 
include detailed questions pertaining to each of the course 
requirements that are required for degree completion. 
Further study is planned to look at how the factors that 
influence enrollment into the program affect the 
program’s outcome. Additionally, we intend to examine 



whether gender has a significant impact on the factors 
that affect the decision to enroll in a post baccalaureate 
business education program. 
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ABSTRACT 

This paper reports the results of a study of student attitudes towards distance education in the US and in Australia. 
Students involved in the study included undergraduate and postgraduate students in the US and postgraduate students 
in Australia. The study used a model for distance education adoption as a basis for the research and the study was 
partly aimed at validating that model. The findings from the study indicate that for most people, particularly 
undergraduates , distance education is not the preferred option. There are however certain students for whom it is 
most attractive and to some extent it seems that these students disagree with the majority who see it as falling short 
of the traditional classroom experience. The study has the limitation of self-selection— further work in different 
disciplines and cultures would be worthwhile. 



INTRODUCTION 

Distance education, particularly “on-line” distance 
education is attracting considerable attention from both 
providers of education and potential students. A paper 
(Dick, 2000) published at IAIM last year noted 
significant similarities between this form of education 
and telecommuting — from the employer (or provider) 
perspective, there is the attraction of a wider pool of 
potential recruits (read potential students) savings on 
facilities and organisational infrastructure, meeting 
demand and changing work practices. From a student 
perspective, the telecommuting advantages of reduced 
travel, flexibility and the time to devote to other 
commitments (work, family etc.) are at least initially 
attractive. The IAIM 2000 paper proposed a model for 
evaluation of the potential adoption of on-line distance 
education. The development of this model is summarised 
in the “Background” section below and the model itself, 



which was conceptually derived from the telecommuting 
literature, is provided as Figure 1 towards the conclusion 
of that section. 

This paper reports the results of a test of the portion of 
the model that relates to the individual student. 

BACKGROUND 

The proposed adoption model (see below) was based on 
the benefits costs and risks associated with distance 
education from the perspective of both the student and 
the educational institution and the enablers, drives and 
constraints (Mokhtarian and Salomon, 1994; Tung and 
Turban, 1996) which provides some insight into the 
factors that are likely to influence the acceptance of this 
form of education. The proposed model included as 
potential benefits or drives: 
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• Reasons associated with travel for educational 
purposes, such as not having to attend on a regular 
basis, may reduce travel costs for the student, 
particularly if long distance travel is involved. In this 
context it should be noted that reduction of living costs 
maybe a significant factor for the potential student. 
Also, this area might be broadened to include those for 
whom travel would be impossible, such as those living 
abroad or in remote areas. 

• Better able to manage one’s own affairs eg. more 
independence, flexibility, control of the physical living 
environment, to pursue personal interests — particular 
relevance perhaps to the post-graduate student in the 
sense of better managing work commitments. 

• The increased the possibility of education for those 
who may be disabled or extensively involved in the 
care of dependent children or other relatives. 

• More attractive to those who might find the campus 
environment threatening or intimidating. 

• To spend more time with one’s family. 

• Campus life offers many distractions for the student; 
while mostly seen as an advantage, some students may 
benefit from the possibility of removing themselves 
from these distractions. 

Against, these advantages, the following potential 

drawbacks merit consideration. 

• More difficult to study at home due to less help 
available, motivational problems, increased family 
conflict and distractions — one might expect these to be 
serious impediments to distance education for many 
people, requiring particular personal attributes for 
them to be overcome. 

• The potential feeling among distance students that 
those with physical access to the academic staff get 
enhanced help and assistance. 

• Missing out on resources and occasional casual work 
to supplement student incomes. 

• Travel is seen as a time for completing assignments, 
reading, study, etc. 

• A significant issue for potential distance students may 
be the need to equip a home study area with a PC and 



appropriate software, telephone line, communications 
software. 

• Missing out on the extra-curricular activities that take 
place on campus could be viewed by many as a serious 
impediment to distance education. 

• Not getting to know one’s fellow students, no easy 
access (formal or informal) to academic staff. At a 
more strategic level, a diminished educational 
experience may result. 

In addition to the above there is a long list of electronic 
enablers which facilitate telecommuting — PCs and 
laptops, printers, modems, copiers, fax machines, 
cellular telephones, answering machines, high speed 
communications links and access to e-mail and the 
Internet (Hotch, 1993; Tung and Turban, 1996). While 
clearly not all are required for educational tasks, this list 
is a useful starting point for the types of electronic 
assistance that would facilitate distance education. At 
present much of this equipment is made available free of 
charge to students in the traditional campus 
environment — considerable expense would be incurred 
by the student in equipping himself with such 
technology. On the other hand many universities are 
moving to requiring (or expecting) students to have such 
technology available at home. 

Parallels were drawn between educational and work- 
place tasks — the understanding of prescribed material, 
assignments, experiences and acquisition of knowledge 
on one hand and the components of a job on the other. 
Using a theoretical task model to encompass the 
component, co-ordinative and dynamic themes of 
complexity (Wood, 1986), the task characteristics of 
uncertainty and equivocality (Daft and Macintosh, 1981) 
and the organisational issues of resources and scheduling 
of work (Thompson, 1967), a set of attributes for 
educational tasks was developed. It was proposed that 
this model form a central component of a research model 
for the evaluation of the suitability of educational tasks 
to distance education. 

Considering the complexity of tasks (Wood, 1986) — in 
general terms as the degree of complexity rises, the task 
becomes less suitable (or more difficult) for distance 
education. Component complexity is a function of the 
number of distinct acts that are required to perform the 
task and the number of information cues to be processed 
in performing these acts. Component complexity is also 
affected by the task being dependent on completion of 
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other tasks. The type of task may have relevance here 
too — some concepts may be difficult to explain or 
demonstrate without “hands on” experience — for 
example dissection, modelling and instrument operation. 
Co-ordinative complexity refers to the form and strength 
of relationships and the sequence of inputs. Wood 
suggests that the more complex the timing, frequency, 
intensity and location requirements, the greater the 
knowledge and skill the individual must have to be able 
to perform the task. Changes in the acts and information 
required or in the relationships between inputs and 
products Wood calls dynamic complexity. This too can 
create shifts in the knowledge or skills required. 

To illustrate, if we consider component complexity, 
tasks with minimal component complexity may be those 
such as reading a study guide, notes or a textbook and 
answering a series of “review” questions. At the other 
end of the scale, research using multiple resources, 
including hard copy and electronic journals, textbooks 
and the Internet, discussions with a colleague and 
writing up a summary of the research may present 
difficulties for the distance education student. Likewise, 
co-ordinative complexity could range from one person 
completing an assignment to working as part of a team, 
with each member responsible for various components 
and then the team having to link them together to 
produce a final product. 

The task characteristics (Daft and Macintosh, 1981; 
Daft, Lengel and Trevino, 1987) of equivocality 
(ambiguous meanings or instructions) and uncertainty 
(about what is required or how to go about it) are 
relevant to tasks involved in distance education 
too — considerable difficulty might be expected to be 
experienced by the student if tasks are not clearly 
explained with no ambiguity and specified to reduce 
uncertainty. 

Similarly, the environment in which the tasks take place 
(Thompson, 1967) may have some relevance to their 
suitability — serial dependence refers to the need to wait 
on others (academic or student) in order to commence or 
complete one’s own work . Also relevant is the degree 
of “networking” and team building that educational tasks 
are designed to include. 

The personal attributes of the individual student would 
seem to have relevance too. These are most likely to be 



in the areas of characteristics such as the ability to get 
information required, knowing when advice is needed, 
the ability to solve one’s own problems and good self- 
management (Venkatesh and Vitalari, 1992; Gray, 
Hodson and Gordon, 1993; Wheeler and Zackin, 1994; 
Mokhtarian and Salomon, 1996) and the home 
environment (Yap and Tng, 1990; Mannering and 
Mokhtarian, 1995). For the distance student, knowing 
where to get relevant information and when to seek 
advice would seem to have particular importance, as 
does the ability to solve his own problems — the added 
reliance on information technology and communications 
equipment gives this aspect added weight. Under- 
graduates are more likely (perhaps than their post- 
graduate counterparts) to have motivational problems 
and will need to develop time management skills to 
enable work of an appropriate quality to be delivered on 
time. On the subject of the household environment, the 
telecommuting issues (Mannering and Mokhtarian, 
1995) of presence of small children, number of people in 
the household and family orientation may also have 
some effect on the preference to study at a distance. 

To some extent the role of the academic is analogous 
with that of the supervisor. As the supervisor controls 
allocation, timing and resources for tasks (Starr, 1971), 
the academic controls task content, timing and the 
required resources and becomes an important point of 
contact and resource for the student. 

Telecommuting literature also provides some pointers to 
demographic influences on the preference to 
telecommute — age, gender, time in the work-place, job 
type, education, transport, presence of small children and 
the number of cars in the household (Mokhtarian and 
Salomon, 1997; Belanger, 1999; Dick and Duncanson, 
1999) — some of these seem to have relevance to the 
decision to engage in distance education. 

The perspective of the educational institution, plus the 
above issues led to the development of a proposed model 
to assess the likely adoption of distance education, 
shown in Figure 1 below. The above issues, which are 
grouped together as a basis for a series of constructs on 
the left-hand side of the model are represented as 
variables in the data collection instrument (see Appendix 
A). The research reported in this paper deals with shaded 
portion of the model only. 
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FIGURE 1 

THE RESEARCH MODEL 



Extra students 
Cost savings, etc. 




In the distance education literature there is considerable 
support for the above issues - accessibility, convenience, 
international (or recognised) instructors and a “consumer 
orientation” (Alavi, Yoo and Vogel, 1997; Emmons, 
1999), and the ability to continue education or keep up 
to date while having only limited time available due to 
heavy work commitments (Jana, 1999; Boisvert, 2000). 
Likewise, many of the potential disadvantages — there is 
broad support for the notion that an educational 
programme is far more than a curriculum and that there 
are benefits from a “surround interaction” between the 
students, the instructor and the lectures. This rich 
variety of interaction is likely to be lost (Bertagnoli, 
2001). Others include not learning the skills to think on 
one’s feet, the absence of support and help, longer to 
develop a rapport between student and professor and cost 
issues related to tuition and technology (Emmons, 1999). 

216 



Attempts to measure satisfaction with distance education 
have been sporadic, other than the measure of 
enrolments and the growth in the number of institutions 
offering some form of distance education. One recent 
approach using the service industry as a base (Long, 
Tricker, Rangecroft and Gilroy, 2000) based the 
assessment largely on immediate application in the work 
place — not in an invalid measure, but perhaps only one 
of many. 

THE DATA 

The data collected for this study comes from five 
sources — although this is the first test of this model and 
to some extent must be considered a pilot study, the 
following groups, reflecting as they do different cultures, 
levels of study and varying experiences with distance 
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education, provide for considerable comparative 
analysis. 

METHODOLOGY 

The researchers designed the initial survey instrument 
after careful consideration of the issues raised in the 
literature and reflected in the above model. A copy of 
the instrument may be found at Appendix A. The basic 
structure of the survey instrument measured perceptions 
in the following areas. 

Section Contents 

1 Advantages of distance education 

2 Disadvantages of distance education 

3 Education related tasks 

4 Ability to undertake distance education 

5 Suitability of distance education 

6 Personal demographics 

To check for ambiguity, its ability to be understood, and 
the amount of time taken to complete, a member of 
academic staff and 4 students completed this survey. No 
modifications were made to the survey instrument as a 
result of the review of the completed surveys. 

It is recognised that the disparate data groups listed in 
Table 1 cannot be considered to form a homogenous 
group for evaluation of the model and its underlying 
constructs, however it is considered that it is acceptable 
to combine the groups for the purposes of instrument 
validation. 



Reliability of the instrument in terms of stability was 
measured by test-retest surveys and in terms of construct 
validity by Cronbach alpha scores to determine internal- 
consistency reliability. This is a generally accepted 
procedure (Judd et al. 1991; Frankfort-Nachmias and 
Nachmias, 1996). There is considerable support for the 
use of the test-retest procedure to ensure that the answers 
are stable and there is minimal noise in the measurement 
process at the individual level. The Cronbach alpha is 
now the preferred measure of internal-consistency 
reliability for construct measurement and is performed 
by analysing the statements in the survey (Judd et al. 
1991). In this study, both were used. 80% of the test- 
retest correlations were greater than .68 and all were 
significant at the .001 level. The Cronbach alpha scores 
ranged between .70 and .81 for each of the constructs, 
falling into the “respectable” to “very good” ranges 
(DeVellis, 1991). 

Validity of the measurement instrument was assessed in 
terms of content validity, (specifically including face 
validity and sampling validity), empirical validity and 
construct validity. This methodology is in accordance 
with generally accepted procedure (Frankfort-Nachmias 
and Nachmias, 1996). Specific procedures conducted to 
assess each of these were: 

• Face validity (a necessarily subjective assessment of 
the instruments’ appropriateness) was assessed and 
achieved by the researchers by comparing the 
instrument with other, similar telecommuting 



TABLE 1 
DATA GROUPS 



Group 


n 


Description 


A 


159 


Predominantly first and second year undergraduate students at a US 
university doing an introductory IS course 


B 


49 


Predominantly final year undergraduate students at a US university doing 
a database course 


c 


18 


MBA-type students at an Australian University doing an IS Management 
course face-to-face on campus 


D 


29 


MBA-type students at an Australian University doing the same course as 
Group C (different instructor) via a web based education package — 
interaction with the instructor was largely limited to this package and 
email. 


E 


31 


MBA students at a US University doing an IS Management course face- 
to-face on campus and others doing the same course (and the same 
instructor) with some students at remote locations interacting via full tele- 
conferencing facilities 
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surveys and questionnaires, in particular studies based 
in the US. 

• Sampling validity (whether a given population is 
adequately sampled by the measuring instrument) was 
provided by the distribution of the survey to all 
members of a class and by the researcher not following 
any particular bias in selection of the students to whom 
to distribute the survey. The large sample size in 
group A goes some way to ensuring sampling validity, 
too. 

• Empirical validity was evaluated both by using 
measures contained within the survey to check the 
consistency of results. 

• Construct validity was assessed by means of Cronbach 
alpha scores. 

In order to assess the effect of each of the independent 
variables on the preference for distance education, linear 
regression was performed on the data. Also a series of 
non-parametric t-tests (Mann- Whitney) was conducted 
to identify variations in the perceptions and preferences 
between the different groups of students in the data. 

RESULTS 

Undergraduate Students 

The data contained two groups of undergraduate US 
students, one group predominantly freshman and 
sophomores doing a face-to-face introductory IS course, 
the other predominantly seniors doing a face-to-face 
database course. It was hypothesised that these two 
groups would see the advantages and disadvantages in a 
similar light, but that the seniors would see more 
complexity, ambiguity and communication requirements 
in their course, which, it was expected, would make such 
a course seen as less suitable for distance education. 

The results of a series of Mann- Whitney tests only 
partially support this hypothesis. While the students do 
see the advantages and disadvantages in a similar light 
(the responses to only two of the statements were 
significant at the .05 level) and the senior students saw 
much greater task complexity, ambiguity and 
communication and interaction requirements (7 items 
significant at the .05 level) the two groups viewed the 
tasks associated with their course as being suited to 



distance education in a similar fashion. In addition this 
group agreed with the statement that while distance 
education was an acceptable instructional delivery 
system, it fell short of the traditional classroom 
experience. 

In terms of the preference to undertake distance 
education (and most of these students disagreed with the 
statement that they would prefer distance education over 
traditional classroom based courses — a mean of 3.52 in 
a 5 point scale from “strongly agree” to “strongly 
disagree”), regression analysis, (using stepwise 
regression, initially computed by running each section 
against the dependent variable, then by running the 
variables that loaded against the independent variable) 
suggests that for this group of students, the significant 
elements (R 2 = .31) are: 

• more help available on campus, 

• distractions at home, 

• not seeing distance education as providing long 
periods of concentration time, 

• a diminished classroom experience, and 

• the absence of computing and communications 
resources in the home. 

Post-graduate Students — Australia 

Two groups of students undertaking the same MBA-type 
course in Australia were included in this study. One 
group of students took the course in a traditional 
environment with a weekly discussion group facilitated 
by a professor, the other group completed the course by 
distance education, which in this context means being 
given a hard-copy of notes and readings (as were the 
face-to-face group) and then interacting with each other 
via an Internet-based discussion group (facilitated by a 
professor) and email. Both groups submitted assignment 
work electronically. Due to nature of the delivery of this 
course and the mix of students involved it is reasonable 
to assume that each group had either some experience or 
some knowledge of the other form of delivery. 

The distance education group saw the advantages and 
disadvantages differently to the classroom based group. 
They felt that the advantages of being able to manage 
work commitments, flexibility and being able to choose 
the time for educational work applied much more 
strongly to them that the classroom based group did. 
They also saw the potential disadvantages of the 
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difficulty of studying at home, better help on campus, 
missing out on the professional interaction, the possible 
diminished classroom experience and difficult to contact 
the instructor all less strongly than the traditional group. 
These differences were all significant at the .05 level. 
The groups showed no differences in the way they 
viewed the task related activities or their abilities to 
undertake distance education. 

Turning to the ways the two groups viewed distance 
education, marked variations were noted. In terms of 
distance education being of lesser quality than traditional 
classroom education, the traditional group agreed, the 
distance group did not (p < .001). The traditional group 
indicated they would only participate in distance 
education if they could not attend campus classes, the 
distance group disagreed (p < .05). The traditional group 
did not see the tasks as suitable for the distance 
education environment, the distance group did (p < .05). 
The traditional group would not encourage others to use 
distance education, the distance group would (p < .05). 
The traditional group did not prefer distance education 
and saw it is as falling short of the classroom experience 
the distance group disagreed with both of these (p < .001 
in both cases). For the distance group, the mean was 
3.18 indicating neutrality. It is also worthy of note that 
the traditional group were less likely to believe they had 
the skills and ability to be a successful distance 
education student. 

For the traditional group, considering the (non) 
preference for distance education (a mean of 4.06 in a 5 
point scale, where 5 = “strongly disagree”) regression 
analysis suggests that for this group of students, the 
significant elements (R 2 = .59) are: 

• missing out on benefits available on campus, and 

• not seeing the ability to choose a time for study as 
important. 

For the distance group, considering the preference for 
distance education (a mean of 2.57 in the 5 point scale,) 
regression analysis (performed as outlined above) 
suggests that for this group of students, the significant 
elements (R 2 = .54) are: 

• being able to concentrate for long periods on course 
related tasks, and 

• not feeling that there is better help available on 
campus. 



Post-graduate Students — US 

The final group included in the study were two MBA 
classes in the US — some were taught face-to-face, others 
in an identical course were taught face-to-face for some 
students while those in outlying areas joined the class by 
means of video-conferencing. Again it is reasonable to 
assume that each group had some knowledge and/or 
experience of the alternative form of delivery. 

This group was the group most opposed to distance 
education — the responses to the statement “I prefer 
distance education over traditional education” gave a 
mean of 4.1 on the 5 point scale. An examination of the 
data indicates that it is bi-modal with 55% choosing 
“strongly disagree” (5) and 30% choosing the middle of 
the range (3). They agreed with the statement that while 
distance education was an acceptable instructional 
delivery system, it fell short of the traditional classroom 
experience. 

Regression analysis (performed as outlined above) 
suggests that for this group of students, the significant 
elements determining the preference for distance 
education (R 2 = .85) are: 

• a diminished classroom experience, 

• not seeing distance education as enabling them to 
better manage work commitments, 

• tasks assigned typically involving many different 
components (complexity), and 

• a preference for working on their own rather than with 
other students. 

Further analysis will be conducted on this group in an 
attempt to explain these general findings, perhaps due to 
the small sample size and the bi-modal distribution. 

CONCLUSIONS, LIMITATIONS, 

AND FURTHER RESEARCH 

For undergraduate students, distance education is very 
much a second option, at least in the US environment. 
While it might suit a small group of students who have 
the skills, resources and perhaps the need to not attend 
campus classes, by and large they will miss the help and 
interaction available there and would incur considerable 
expense in setting up the necessary computing and 
telecommunications equipment. 
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The Australian group clearly demonstrates different 
perceptions of distance education, based on one’s desire 
for it. The findings are almost completely reversed for 
the two groups involved in the one course — one group 
who have chosen to study by distance have chosen it for 
almost the same reasons as the other group chose to 
study in a more traditional manner, just a different 
perception of these reasons. 

It is perhaps worthy of note that there is a widespread 
belief that one should not have to pay as much for 
distance education as for traditional campus based 
education and that universities see distance education as 
attractive because it provides additional revenue without 
the need for additional resources. This finding holds 
true for even the group of Australian distance students 
who expressed a preference for distance education. 

An obvious limitation of this study has been the degree 
of self-selection in the data. Clearly, the students 
involved in the study have chosen courses that suit them, 
their skills and their attributes. However, this limitation 
in itself is useful, demonstrating that students will 
choose the best option for themselves and adding weight 
to the belief or conjecture that not all courses are suited 
to all types of students. 

The study does support the model to a considerable 
degree. The high R 2 scores from the regression analysis 
suggest that the measurement items are valid for the 
constructs and that the constructs being tested lead to 
determining the preference for distance education. More 
work is necessary in this area, covering a wider range of 
courses, cultures and students. The authors would 
welcome collaborating with other researchers to conduct 
studies in other disciplines and countries. 
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APPENDIX A 

DISTANCE EDUCATION SURVEY 

1 . Please indicate the extent to which you agree that the following potential advantages of distance education apply 
to you: 







Strongly 


-> -> - 




->-> - 


Strongly 






Agree 








Disagree 


a. 


Distance education allows me to reduce travel and 
commuting costs. 


□ 1 


□2 


□3 


□4 


□ 5 


b. 


Distance education allows a reduction in living costs 
due my ability to live at home, not on campus. 


□ 1 


□2 


□3 


□4 


□ 5 


c. 


Distance from home would make class attendance on 
campus impossible. 


□ 1 


□2 


□3 


□4 


□5 


d. 


I am better able to manage work commitments by not 
being required to attend class on campus. 


□ 1 


□2 


□3 


□4 


□5 


e. 


Distance education enables me to complete class work 
if disabled or taking care of dependents. 


□ 1 


□2 


□3 


□4 


□5 


f. 


I find the campus environment intimidating or 
undesirable. 


□ 1 


□2 


□3 


□4 


□5 


g- 


Personal reasons such as family, flexibility make 
distance education attractive to me. 


□ 1 


□2 


□3 


□4 


□5 
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Strongly 

Agree 


-> -> - 




->-> - 


Strongly 

Disagree 


h. Fewer distractions for me at home allows me to be a 
productive distance education student. 


□ 1 


□2 


□3 


□4 


□5 


i. Being able to choose the time to study and work on 

assignments is important to me. 


□ 1 


□2 


□3 


□4 


□5 



Please indicate up to 3 of the above that you see as the most and least important: 

Most important Least important 
1 st 



3 rd 



2. Please indicate the extent to which you agree that the following potential disadvantages of distance education 
apply to you: 







Strongly 


-> -> - 




->-> - 


Strongly 






Agree 








Disagree 


a. 


I would find it more difficult to study at home due to less 
help, motivational problems, increased family conflicts, 
distractions. 


□ 1 


□2 


□3 


□4 


□5 


b. 


I feel that there is better help for me available on campus. 


□ 1 


□2 


□3 


□4 


□5 


c. 


I would miss out on benefits available on 
campus — resources, possible employment, etc. 


□ 1 


□2 


□3 


□4 


□5 


d. 


The cost of procuring the necessary distance education 
equipment for my home would be expensive for me. 


□ 1 


□2 


□3 


□4 


□5 


e. 


I would miss out on the extra-curricular activities 
available on campus. 


□ 1 


□2 


□3 


□4 


□5 


f. 


Missing out on the professional interaction with one’s 
fellow students would be a concern to me. 


□ 1 


□ 2 


□3 


□4 


□5 


g- 


There would be a diminished classroom experience — less 
discussion, interaction with professors. 


□ 1 


□2 


□3 


□4 


□5 


h. 


I find it very difficult to contact the instructor. 


□ 1 


□2 


□3 


□4 


□5 



Please indicate up to 3 of the above that you see as the most and least important: 



1 st 
2 nd 
3 rd 



Most important Least important 
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3. Thinking about your academic work, please indicate your degree of agreement with the following statements as 
they typically concern course related tasks you have to complete (study, assignment work, exam preparation etc.): 







Strongly 


-> -> - 




->-> - 


Strongly 






Agree 








Disagree 


a. 


The final product of the tasks that I am assigned 
typically involves the completion of many different 
components. 


□ 1 


□2 


□3 


□4 


□5 


b. 


My tasks often require me to work with fellow students. 


□ 1 


□ 2 


□3 


□4 


□5 


c. 


Sometimes the task deliverables change over the duration 
of the assignment (e.g., the instructor adds or deletes one 
or more components). 


□ 1 


□2 


□3 


□4 


□5 


d. 


Tasks I am given are not always clear and may be 
interpreted in different ways. 


□ 1 


□2 


□3 


□4 


□5 


e. 


I am often uncertain about what to do to complete the 
final product. 


□ 1 


□2 


□3 


□4 


□5 


f. 


The tasks I am assigned are often dependent on at least 
one other student completing his work first. 


□ 1 


□2 


□3 


□4 


□5 


g- 


The task deliverables are clear, but can be accomplished 
in a number of ways. 


□ 1 


□2 


□3 


□4 


□5 


h. 


I would typically rather work on my own, than with other 
students. 


□ 1 


□2 


□3 


□4 


□5 


i. 


The tasks I am assigned require minimal resources (e.g. 
software, library, etc.). 


□ 1 


□ 2 


□3 


□4 


□5 


j- 


The tasks I am assigned allow me to work at my own 
pace. 


□ 1 


□ 2 


□3 


□4 


□ 5 


k. 


Assigned tasks require long periods of concentrated 
attention. 


□ 1 


□ 2 


□3 


□4 


□ 5 


1. 


Distance education enables me to concentrate on course 
related tasks for long periods. 


□ 1 


□ 2 


□3 


□4 


□ 5 


m. 


“Due dates” for tasks assigned are clearly stated. 


□ 1 


□ 2 


□3 


□4 


□ 5 


n. 


There is a need for a considerable degree of 
communication with my fellow students. 


□ 1 


□2 


□3 


□4 


□5 


0. 


There is a need for a considerable degree of 
communication with academic staff. 


□ 1 


□2 


□3 


□4 


□ 5 



4. Thinking about your ability to study via distance education, please indicate your degree of agreement with the 
following statements: 





Strongly 


-> -> - 




->-> - 


Strongly 




Agree 








Disagree 


I am capable of making good decisions about the tasks I 
am assigned. 


□ 1 


□2 


□3 


□4 


□5 


I know where to get the relevant information I need to 
complete the assigned tasks. 


□ 1 


□2 


□3 


□4 


□5 
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Strongly 

Agree 


-> -> - 




->-> - 


Strongly 

Disagree 


c. 


I have no difficulty determining when I should seek 
advice. 


□ 1 


□2 


□3 


□4 


□5 


d. 


I am good at self management, possessing the motivation, 
time management, etc. that is needed to deliver quality 
work on time. 


□ 1 


□ 2 


□3 


□4 


□5 


e. 


I have the computing and communications resources I 
need to be an effective distance education student. 


□ 1 


□2 


□3 


□4 


□5 


>. Please indicate the extent of your agreement with the following statements: 














Strongly 

Agree 


-> -> - 




->-> - 


Strongly 

Disagree 


a. 


Distance education is of lesser quality than traditional 
class-room-based campus education. 


□ 1 


□2 


□3 


□4 


□5 


b. 


I only participate in distance education because I can’t 
attend campus classes. 


□ 1 


□2 


□3 


□4 


□5 


c. 


The tasks associated with my course are suitable for the 
distance education environment. 


□ 1 


□2 


□3 


□4 


□5 


d. 


I should not have to pay as much for distance education 
as for traditional campus based education. 


□ 1 


□ 2 


□3 


□4 


□5 


e. 


I would encourage most professionals to participate in 
distance education. 


□ 1 


□ 2 


□3 


□4 


□5 


f. 


Instructors should not assign the same tasks to distance 
based students as they assign to campus based students. 


□ 1 


□ 2 


□3 


□4 


□5 


g- 


Distance education courses are designed with the distance 
student in mind. 


□ 1 


□ 2 


□3 


□4 


□ 5 


h. 


I believe I have the skills and ability to be a successful 
distance education student. 


□ 1 


□ 2 


□3 


□4 


□5 


i. 


Distance education is attractive to Universities because it 
provides additional revenue without the need for 
additional resources. 


□ 1 


□2 


□3 


□4 


□ 5 


j- 


I prefer distance education courses over traditional 
classroom based courses. 


□ 1 


□2 


□3 


□4 


□5 


k. 


Distance education is an acceptable instructional delivery 
system, but it falls short of the traditional classroom 
experience. 


□ 1 


□ 2 


□3 


□4 


□5 



6. Please provide the following general information about yourself: 
1 . Are you : 

□ Male □ Female 



2. In which age group are you? 

□ 20 or younger □ 21-25 □ 26- 30 □ 31-40 

□ 41-50 □ 51-60 □ over 60 
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3. Postcode/Zip code 



Home 



Work (or country if overseas 



.) 



4. What is your education level? {Indicate highest level , excluding current courses of study ) 

□ High school □ Technical certificate or diploma 

□ University graduate □ Post-graduate □ Other (please specify) 

Year completed? 

5. Which of the following best describes your current work? 



□ 


Manager/Administrator 


□ 


Administrative support 


□ 


Professional 


□ 


Technical 


□ 


Supervisor 


□ 


Clerical 


□ 


Services/Repair 


□ 


Customer enquiry 


□ 


Sales/Marketing 


□ 


Education 


□ 


Other (Please specify) 







6. How many people in your household (including yourself) fall into each of the following age groups: 

Under 2 years old 2-5 years old 6-15 years old 

16-20 years old 21-65 years old over 65 years old 
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ABSTRACT 

The researchers, using a variation of the SERVQUAL instrument, repeated a 1999 study to measure students’ 
satisfaction with instructional technology tools used in their classrooms. Student satisfaction varied by course 
discipline, by instructional technology, by anticipated grade, and by frequency of use . Female respondents were less 
satisfied than male respondents. Satisfaction generally rose with frequency of use. There are significantvariations 
of satisfaction by discipline and technology choice, but little interaction effect. Factor analysis did not reveal the five 
hypothesized dimensions of SERVQUAL. Overall results were generally cons istent with the 1999 study. 



INTRODUCTION 

The researchers, using a variation of the SERVQUAL 
instrument, repeated a 1999 study and measured 
students’ satisfaction with a broad spectrum of 
classroom technology tools. Classes in five academic 
courses Introductory MIS, Business Communication, 
Principles of Economics, Principles of Marketing, and 
Accounting Information Systems in AACSB- 
accredited schools nationwide, participated. 

Purpose 

Higher education institutions across the country are 
scrambling for competitive advantage, whetherthrough 
distance education offerings or in their traditional ‘‘brick 
and mortar classrooms. High service quality is 
necessary to protect competitive advantage. Student 
satisfaction with their classroom experiences plays an 
important part in contributing to aschool’s competitive 
advantage. This research replicates a 1 999 study (Kleen, 



Shell, and Cox, 1999) of a cross-disciplinary student- 
centered assessment of satisfaction with instructional 
technology being used in the business classroom. The 
1 999 study used an instrument inspired by SERVQUAL, 
a widely used instrument to measure customer service 
quality. The undercurrents ofthe research include such 
academic problems as limited technology dollars, 
questionable relevance of industry training room setups 
as models, and the need for classrooms to support 
several faculty with different teaching styles and 
different technology needs. This research is also 
inspired by the need to evaluate new and evolving 
technologies with uncertain cost-benefit ratios and 
relatively untested classroom impact. 

The long-range intent of the researchers is to provide 
faculty with evidence to help them choose instructional 
tools appropriate to their classes. The current 
researchers elected to conduct the 2001 study for two 
reasons: (1) to review technologies afterthree years and 
compare whether students are now more satisfied or less 
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satisfied with the technologies used in the classrooms 
than in 1999, and (2) to corroborate the results of the 
Kleen, Shell, and Cox 1999 study. 

The Kleen, Shell, and Cox 1999 study revealed that 
students in the various disciplines did have different 
levels of satisfaction with various technologies such as 
overhead transparencies, computer slide shows, software 
demonstrations, and student in-class computer activities. 
The authors found that satisfaction varied little by gender 
or by respondent age group, although satisfaction varied 
with intensity of technology use in some ways. In a 
related study by Kleen, Shell, and Zachry (2001), the 
researchers found that male students in AIS classrooms 
were uniformly less satisfied than female students with 
certain technologies. 

Even though the project is a replication of an earlier 
study, the research questions are cross sectional. What 
are the top rated satisfaction items for the various 
technologies? What are the satisfaction differencesacross 
course disciplines? Are there interactions by gender? 
How satisfied are thestudent consumers of instructional 
technology being used in the business classroom? When 
these modern technologies are used in the classroom, do 
students believe they learn more? Do they believe they 
pay attention better? Do they believe they become more 
confident about their learning? Do they believe they 
understand more? In the conclusions, parallels will be 
drawn between the 1999 study findings and the 2001 
findings. 

Literature Review 

To date, a very limited amount of empirical research 
exists related to how students benefit (or perceive they 
benefit) from faculty use of technology tools and/or 
methods such as electronic slideshows, Internetactivities, 
live software demonstrations, and hands-on student 
computer activities within the actual classroom. 

The 22-item, two-part scale called SERVQUAL, 
originated by Parasuraman, Zeithaml, and Berry (1988), 
has become a widely used tool for measuring customer 
service quality. Buttle (1996) emphasized that service 
quality can be best assessed from the point of view of 
consumers; this is the approach taken by SERVQUAL. 
Kettingerand Lee(1997) noted its practical valueas both 
a benchmarking tool within an industry and as a 
diagnostic tool. While SERVQUAL studies have 
covered many service sectors, Buttle (1996) found no 
articles thatapplied SERVQUAL to a classroom setting. 



Modifications and adaptations of the SERVQUAL 
instrument are widespread; in fact, even the instrum ent’s 
originators have tested modified versions (Parasuraman, 
Zeithaml, and Berry, 1996). In 1997 Van Dyke, 
Kappelman, and Prybutok concluded that using a single 
measure of service quality across industries is not 
feasible, and that future research should involve 
development of industry-specific measures of service 
quality . 

Kleen, Shell, and Cox (1999) focused only on 
perceptions of students as consumers of classroom 
technology, eliminating the use of gap-based scores of 
perception minus expectations (P-E gap scores) of the 
original SERVQUAL. This use of only part of the 
SERVQUALstructure,such as omitting expectations but 
leaving perceptions, is supported by Teas (1994) and 
Brown, Churchill, and Peter (1993). Additionally, Van 
Dyke, Kappelman, and Prybutok (1997) note the IS- 
adapted SERVQUAL instrument, utilizing difference 
scores, is neither a reliable nor a valid measurement for 
operationalizing the service quality construct for an 
information systems services provider. 

The research and debate over the use of IS-adapted 
SERVQUAL instruments in the information systems 
construct continues (Kettinger and Lee, 1997; Pitt, 
Watson, and Kavan, 1997; VanDyke, Prybutok, and 
Kappelman, 1999). The research and debate over the use 
of SERVQUAL-inspired instruments in the higher 
education construct should do no less. 

Methodology 

The researchers repeated an earlier study that used an 
instrument containing 19 questions inspired by 
SERVQUAL. These items are not “difference-score 
based or “gap-based, but rather focus only on the 
perceptions of students as consumers of classroom 
instructional technology. They do not include student 
expectationsof desired or adequate service. The original 
19 satisfaction items on the questionnaire remained the 
same as the Kleen, Shell, and Cox 1999 study. The 
researchers added an explanatory variable of self-reported 
projected final grade in the course to the other 
classification variables of major, age, and gender. The 
1999 study containedthe classification variable, “student 
classification. This variable was not used in the 2001 
study for its lack of explanatory power. Students in 
AACSB schools were the target population. The current 
research project received approval from the researcher 
University’s Human Subjects Institutional Review Board. 
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The researchers conducted a systematic random sample 
of one-third of the approximately 350 AACSB- 
accredited schools. Department headswere contactedfor 
each of five business courses selected for the study. 
Specific courses selected included the following: 
Management Information Systems, Business 
Communication, Principles ofEconomics, Principlesof 
Marketing, and Accounting Information Systems. Each 
department head received a cover letter from the 
researchers and letters of invitation to distribute to the 
faculty teaching the course identified in the letter. 
Because of the known wide variance in the departments 
responsible forbusiness communication, these materials 
were sent to deans’ secretaries for routing to involved 
faculty. Faculty members who wished to participate 
after reading the letter of invitation e-mailed course 
name, number, and number of students to the 
researchers. The researchers then mailed survey 
instruments and response forms for each student in the 
classroom of participating faculty in April. Faculty 
members were encouraged to administerthe survey near 
the end of the term. 

The current study used only four of the seven 
instructional technologies of the 1999 study. The 
categories of overhead transparencies, videotape or 
television programs, and CD-ROM or multimedia 
presentations were eliminated from the current study 
because two were rarely used in 1999. The current 
researchers also wanted to focus on only those 
technologieswith an electronic component. Theprimary 
focus of the instrument was on the four remaining 
classroom instructional technologies: 

electronic slide shows 
live software demonstrations 
live Internet connections 
student in-class computer activities 

Participating instructors selected two of the four 
technologies used in their classes and directed students 
to answer the question sets for those specific 
technologies. 

Parallel sets of 20 questions were presented for each 
instructional technology. Eighteen of the twenty 
questions were built around the five SERVQUAL 
dimensions of Tangibles, Reliability, Responsiveness, 
Assurance, and Empathy. Students responded based on 
a five-point Likert-type scale, assumed by the 
researchers to generate interval data. The 19 th item was 



a global item. The final question (20) asked the student 
to estimate in what percentage of the class meetings a 
particular technology was used so that the researchers 
could determine how satisfaction varied with intensity of 
use. 

The 20 questions students responded to for each 
technology are listed below. Students responded to a 
five-point scale, with answer options ranging from a 
strongly agree (1) to a strongly disagree (5). 

1. The use of made it easy to see the 

material presented. 

2. The use of was not appropriate in this 

class. 

3. The use of helpedme learn the material 

presented. 

4. The use of was a good way to reinforce 

assigned reading material. 

5. worked when it was 

supposed to. 

6. The use of helped me pay attention in 

class. 

7. The use of was distracting. 

8. The use of helped make this course more 

interesting. 

9. The use of allowed me to focus better on 

what the professor was saying. 

1 0. The use of w as more trouble than it was 

worth. 

11. The use of made me more confident 

about what I was learning. 

12. The use of helped me to better 

understand fundamental course concepts. 

13. The use of tended to oversimplify 

lectures. 

14. The use of helped me “keep up with 

lecture. 
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15. The used in this class was 

visually appealing. 

16. The use of helped me organize my 

lecture notes. 

17. The use of helped me understand the 

lecture material. 

18. is a modern, up-to-date 

lecture method. 

19. I wish more of my instructors used in 

their classes. 

20. was used in approximately 

percent of the meetings of this class. 

Items 2, 7, 10, and 13 were negative in nature. For 
purposes of analysis, absolute scores were invertedto be 
comparable to positively worded items. 

Descriptive statistics for the 2001 data will answer a 
variety of investigative questions. Descriptive statistics 
included the mean scores for each ofthe 19 satisfaction- 
based items, in each of the technology groups, in the 
aggregate, and in various subgroups such as mean scores 
by gender, by course discipline, and by anticipated 
grade. Descriptive statistics also included regression as 
a descriptive device. These descriptive statistics were 
used to illustrate: 

Which items get higher mean scores as they are used 
more frequently? Which get lower? 

Which satisfaction item (among 1 18) showed the 

highest satisfaction in each technology group? Which 
the lowest? 

Are the top rated items the same based on gender, 
technology choice, and course discipline? 

Do technologies affect user satisfaction in the same 
manner across the five course disciplines? 

Do any items (among 1-18) get highermean scores as 
self-reported grade is higher? 

Do any technologies score betteras self-reported grade 
is higher? 

The researchers also constructed hypothesis tests to 
answer each of the investigative questions above. The 
influence of grade and of usage on satisfaction was 
tested by simple and multiple regression. The impact of 
satisfaction of discipline and technology was tested by 
one-way ANOVA and two-way ANOVA. Hypotheses 



were constructed (overall and grouped by course 
discipline for each of the 18 items) to determine 
whether: 

Student satisfaction varies with frequency of 
instructional technology tool use 
Student satisfaction varies with gender 
Student satisfaction varies with expected grade 
Student satisfaction varies with course discipline 

Factor analysis enabled the researchers to determine 
whether the clustering of 1 8 items matched their a priori 
expectationsfor classification into the five hypothesized 
dimensions. 

FINDINGS 

The findings section ofthe paper begins with descriptive 
statistics, followed by regressions, one- and two-way 
ANOVAs, and factor analysis. 

Descriptive Statistics 

A total of 280 students from 11 classrooms provided 
responses for the study. Class size ranged from 20 to 76. 
Seventy-seven percent of those indicating age were 
under 25 years old; 22.1% were 25 or older. Of those 
respondents indicating gender, 47.5% were male, and 
52.5% were female. Of those indicating their anticipated 
grade at end of course, 37.7% anticipated an A, 35.3% a 
B, 1 3.5% a C, 2.2% a D, and 0.9% an F. On a 4.0 scale, 
that is an average of 2.86 . * 

Electronic slideshow technology had the greatestnumber 
of responses (243 stude nts, 47.5% of all responses in the 
study). Items for live software demonstration tech- 
nology were answered by 85 respondents, 16.6% of the 
total. Eighty-one students (15.8%) answered items for 
live Internet connections, and 103 students (15.8%) 
answered items for student in-class computer activities. 
The cross tabulation of response frequencies, by course 
disciplineandtechnology choice, is illustrated inTable 1. 

Of the 512 total responses (most of the 28 0 students 
responded to two technologies), 64 (12.5%) were from 
students in business communication classes; 74 (14.5%) 
were in a principles of economics class. Principles of 
marketing contributed 73 responses (14.3%); AIS, 53 
responses (10.4%); IS courses, 248 responses (48.4%). 

“Made it easy to see the material presented received the 
lowest mean score (1.89), which provided the highest 
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satisfaction over all technologies, all course disciplines, 
and all respondents. “Modern, up-to-date lecture 
method was next highest satisfaction with a mean score 
of 1.96. The items with worst mean score (lowest 
satisfaction) overall included “tended to oversimplify 
lectures, (2.74), and “helped me organize my lecture 
notes, (2.62). 

Female respondents had higher average scores thanmale 
students, indicating lower satisfaction in 17 of the 19 
satisfaction items. 

In the business communication discipline, the item w ith 
the lowest mean score was “modern, up-to-date lecture 
method (2.02). In business communication the item 
that scored lowest satisfaction was “tended to 
oversimplify lectures (2.89). In economics, the item 
that scored best was “made it easy to see the material 
presented (1.66); the item that scored lowest 
satisfaction was “helped me organize my lecture notes 
(2.69). Within the marketing discipline, the item “made 
it easy to see the material presented (1.86) earned the 
highest satisfaction score. “Tended to oversimplify 
lectures earned the lowest satisfaction score (3.17). 
Within AIS, the highest satisfaction was for “modern, 
up-to-date lecture method (2.1 1); the lowest satisfaction 
was for “helped me organize my lecture notes (2.87). 
Within the IS discipline, the highest satisfaction was for 
“made it easy to see the material presented (1 .83); the 
lowest satisfaction score was for “helped me organize 
my lecture notes (2.70). 

When each technology was examined, differences also 
appeared for highest and lowest satisfaction items. For 
electronic slideshows, the item that had the highest 
satisfaction level was “made it easy to see the material 
presented (1.71); the item that scored worstwas “tended 
to oversimplify lectures (2.80). For live software 
demonstrations, the highest satisfaction item was “made 
it easy to see the material presented (2.01); the item that 
scored worst was once again “tended to oversimplify 



lectures (2.70). For live Internet bes t was once again 
“made iteasy to seethematerial presented (2.07); worst 
was a tie between “tended to oversimplify lectures and 
“helped me organize my lecture notes (2.80). The 
highest satisfaction item for student in-class computer 
activities was “modern, up-to-date lecture method 
(2.03). The lowest satisfaction item for this technology 
was “helped me organize my lecture notes (2.83). 
While these measures are descriptive, not test results, 
they point out that instructors should consider their 
discipline as a strong factor in choosing appropriate 
technology in their classroom. 

Two-Factor Descriptives 

In this section, the descriptive statistics are reported as a 
two-factor model of discipline and technology choice. 
This model does not include the effects of gender, of 
usage rate, or of grade. 

These descriptive statistics use a model which has a 
general effect, a discipline effect, a technology effect, 
and an interaction effect. These effects are additive. 

Satisfaction score = general effect + discipline effect + 
technology effect + interaction 

This equation will be used to illustrate best case and 
worst case scores for two of the 18 satisfaction-based 
items. For item 3, “helped me learn the material 
presented, the best case is 

2.107 - .610 (economics discipline) + 0 (in-class 
computer activities) = 1.497 (satisfaction). 

The corresponding worst case is 

2. 107 + .393 (AIS) + .7 19 (live software demonstrations) 
= 3.219. 
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TABLE 1 

NUMBER OF RESPONSES BY INSTRUCTIONAL TECHNOLOGY AND COURSE DISCIPLINE 



Technology 


Course Discipline 


Business 

Communications 


Principles of 
Economics 


Marketing 


Accounting 

Information 

Systems 


Information 

Systems 


Totals 


Electron ic 
Slideshows 


32 


37 


48 


25 


101 


243 


Live Software 
Demonstrations 


0 


37 


25 


0 


23 


85 


Live Internet 
Connections 


0 


0 


0 


14 


67 


81 


Student In-Class 
Computer Activities 


32 


0 


0 


14 


57 


103 


Totals 


64 


74 


73 


53 


248 


512 



Similar applications for item 17, “helped me understand 
the lecture material, produce 

2.286 - .478 (marketing) + 0 (in-class computer 
activities) = 1.808 (best case) 

2. 286 + .2 1 4 (AIS) + .453 (live software demonstrations) 
= 2.953 (worst case). 

Impact of Anticipated Grade on Satisfaction 

The researchers conducted a regression analysis on 
satisfaction scores of each of the 18 items for each of the 
four technologies, where the independent variable was 
anticipated grade. The vast majority of all such 
regressions showed no significance. However, the 
technology choice student in-class computer activities 
contained virtually all instances of significance. A 
typical example follows: For “helped me understand 
fundamental course concepts, the resulting regression 
equation is satisfaction = 1.601 + .44 2 x anticipated 
grade (sig. = .004, r 2 = .084). In all instances of 
significance, the regression slopes were positive, 
meaning lower grade anticipated equaled lower 
satisfaction. 

Impact of Technology Usage Rate on Satisfaction 

Item 20 contained a five-valued usage variable. A 
simple regression of each of items 1 through 18 was 
conducted with item 20 as the independent variable. 
This was repeated separately for all four technology 
choices. In all cases where the regression was 
significant, the calculated slope was negative, meaning 
higher usage led to higher satisfaction. While signi- 



ficant, the explanatory power of these models is 
limited r 2 were typically under 10% for these models. 
All items for which the regression was significant at the 
.05 level or better are identified in Table 2. 

Does Satisfaction Vary by Technology? 

The following section describes a one-way ANOVA to 
test the equality of mean satisfaction scores across the 
four instructional technologies. The fundamental hypo- 
thesis is 

H 0 = Mean satisfaction score for an item is the same 
across all technology choices. 

H a = Atleast one mean satisfaction score differs from 
the others. 

This hypothes is applied for all the 1 8 satisfaction items. 
H o was rejected for items 1, 2, 4, 5, 7, 9, 1 0, 14, 16, 1 7, 
and 18. In these cases, post hoc comparisons were done 
to determine which mean (s) differed. 

For the item “made it easy to see material presented, 
electronic slideshow (mean = 1.71) scored higher 
satisfaction than both student in-class activities (2.08) 
and live Internet connections (2.08). A low score 
reflects higher satisfaction. No other paired differences 
were significant. 

For “appropriate technology for the class, the only 
significant paired comparison revealed live software 
demonstrations (2.49) were viewed less appropriate for 
the class than electronic slideshows (2.00). 
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For the item “good way to reinforce assigned reading 
material, electronic slideshows (2.25) scored higher 
satisfaction than live software demos (2.57). No other 
paired differences were significant. 

For the item “worked when it was supposed to, 
electronic slideshows (2.06) scored higher satisfaction 
than live Internet connections (2.55) and student in-class 
computer activities (2.39). For the item “technology 
worth the trouble, the only significant paired 
comparison revealed live Internet (2.58) was viewed 
with less satisfaction than electronic slideshows (2.20). 

For the item “helped me keep up with lecture, 
electronic slideshows (2.24) scored higher satisfaction 
than live Internet (2.58). For the item “helped me 
organize my lecture notes, electronic slideshows (2.42) 
scored higher satisfaction than live Internet (2.80) and 
student in-class computer activities (2.83). For the item 
“helped me understand the lecture material, electronic 
slideshows (2.33) scored higher satisfaction than live 
Internet (2.69). 



For “distracting technology, “allowed me to focus 
better on what the professor was saying, and “modern, 
up-to-date lecture method, ANOVA indicated not all 
items were equal, but post hoc comparisons did not 
reveal which. 

Does Satisfaction Vary by Course Discipline? 

The following section describes a one-way ANOVA to 
test the equality of mean satisfaction scores across the 
five course disciplines. The fundamental hypothesis is 

H 0 = Mean satisfaction score for an item is the same 
across all course disciplines. 

H a = At least one mean satisfaction score differs from 
the others. 

This hypothesis app lied for all 18 satisfaction items. H 0 
was rejected for the following items: 1, 2, 3, 5, 9, 10, 
13, 15, 16, and 17. On these items follow-up tests were 



TABLE 2 

TECHNOLOGY USAGE RATE AND ITS IMPACT ON SATISFACTION 



Item 


Electron ic 
Slideshows 


Live Software 
Demonstrations 


Live Internet 
Connections 


Student In-C lass 
Computer 
Activities 


1. Made it easy to see material presented 


X 




X 


X 


2. Technolo gy appropriate for class 


X 




X 


X 


3. Helped me learn material presented 


X 




X 


X 


4. Good way to reinforce assigned readings 


X 






X 


5. Worked when suppo sed to 






X 




6. Helped m e pay attention in class 






X 


X 


7. Distracting technology 


X 








8. Helped make course more interesting 








X 


9. Allowed me to focus on what professor 
said 








X 


10. Technology more trouble than it was 
worth 


, X 








1 1 . Made me m ore con fident abo ut what I 
was learning 




i 




X 


12. Helped me better understand course 
concepts 








X 


13. Tended to oversimplify lectures 








X 


14. Helped me keep up with lecture 








X 


15. Technology visually appealing 


X 






X 


16. Helped me organize lecture notes 








X 


17. Helped me understand lecture material 


X 






X 


18. Modern, up-to-date method 


X 






X 



232 



Proceedings of the 16 h Annual Conference of the International Academy for Information Management 




BEST COPY AVAILABLE 



conducted to see which course disciplines scored better 
or worse than others on each specific item. 

Paired comparisons for item 1, “made it easy to see 
material presented, revealed that the AIS mean (2.34) 
was differentfrom the mean for principles of economics 
(1.66) and the mean for information systems (1.83). 
Therefore, AIS students were much less satisfied than 
economics or IS students on this item. While the result 
was statistically significant, the researchers urge that the 
finding be treated with caution as AIS was the smallest 
of the responding groups. 

For item 2, “appropriate technology for the course, 
marketing students (2.68) were less satisfied than 
economics students (2.01) and information systems 
students (1.93). Information system s students were more 
satisfied than AIS students (2.4 9). 

For item 3, “helped me learn the material presented, 
AIS students (2.68) were lesssatisfied than information 
systems students (2.24). 

For “worked when it was supposed to, economics 
students (1.85) were more satisfied than all other 
disciplines (all 2 .23 or greater). 

For “tended to oversimplify lectures, the marketing 
respondents were less satisfied than students in 
economics or information systems. For “helped me 
organize lecture notes, marketing students (2.23) 
differed from information systems (2.70) and AIS 
students (2.87). In this instance, marketing students were 
more satisfied than the other two groups. 

For “allowed me to focus better on what the professor 
was saying, “worth the trouble, “visually appealing, 
and “helped me understand the lecture m aterial, while 
ANOVA indicated not all means were equal, the post 
hoc comparisons did not reveal a distinguishable pair of 
means. 

Factor A nalysis 

The researchers used the rotated component matrix 
within SPSS’s data reduction tools. Using the same a 
priori listing as in Kleen, Shell, and Cox (1999), the 
researchers assigned the items to the following 
dimensions: Reliability, items 2, 5, and 10; 

Responsiveness, items 8, 9, 13, and 14; Empathy, items 
12, 16, and 17; Tangibles, items 1, 7, 15, and 18; and 
Assurance, items 3, 4, 6, and 1 1. 



Within the technology electronic slideshows, data 
reduction revealed threefactors. These factors loaded as 
follows: Factor I included items 3, 4, 6, 8,9, 1 1, 12, 14, 
15, 16, 17, and 18. Factor II contained items 1,2, 3, 4, 
5, 9, and 18. Factor III contained items 2, 7, 10, and 13. 
Some items loaded onto more than one factor. 

The technology live software demonstrations also 
revealed three factors, which loaded as follows : Factor 

I contained items 3,4, 6, 8, 9, 1 1, 12, 14, 15, 16, and 17. 
Factor II contained items 1, 5, 15, and 18. Factor III 
contained items 2, 7, 10, and 13. 

Within the technology live Internet connection, four 
factors were identified, loading as follows: Factor I 

contained items 3,4, 5,8, 9, 11, 12, 16, and 17. Factor 

II contained items 1, 2, 6, 11, 15, and 18. Factor III 
contained items 7, 1 0, and 13. And Factor IV contained 
item 14 only. 

For student in-class computer activities, four factors 
emerged. These loaded as follows: Factor I contained 
items 1, 2, 3, 4, 8, 12, and 14. Factor II contained items 
5, 6, 9, 11, 14, 15, 16, 17, and 18. Factor III contained 
items 5, 10, and 13. Factor IV contained items 2 and 7. 

For electronic slideshows all items of the Assurance 
dimension were in Factor I. Three of the 
Responsiveness items were also in Factor I. All three 
Empathy items were in Factor I. Two of the four 
Tangibles items were in Factor I. Reliability was 
divided between Factors II and III. 

For live software demonstrations, all the Assurance 
dimension items loaded in Factor I; likewise, all the 
Empathy dimension items loaded in Factor 1 . Three of 
the four Responsiveness dimension items also loaded in 
this factor. Factor II contained two of four Tangibles 
items. Reliability was again sp lit between Factors II and 
III. 

For live Internet connections, three of four items in the 
Assurance dimension loaded in Factorl. Once again, all 
the Empathy dimension item s loaded in this same factor. 
Two of the four Responsiveness items also loaded in this 
factor. Three of four items in Tangibles loadedin Factor 
II. Reliability’s three elements were divided- among 
three factors. The third factor was not associated with 
any dimension. 

For student in-class computer activities, Factor I 
contained two of four Assurance items and two of the 
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four Responsiveness items. Two ofthree Empathy items 
loaded in Factor II; two of four Tangibles items also 
loaded in that factor. Two of the three Reliability items 
loaded in Factor III. 

In no case did the data reduction identify five factors, as 
suggested by SERVQUAL theory. The Empathy and 
Assurance dimensions often combined into the same 
factor. These two dimensions were also well identified 
in Kleen, Shell, and Zachry (200 1). Otherwise, the data 
reduction results were inconclusive. 

CONCLUSIONS AND RECOMMENDATIONS 

The researchers attempted to identify factors affecting 
student satisfaction with classroom instructional 
technologies. The study was a replication of 1999 
research, with the same instrument, but slightly different 
instructions, and with different sampling methods. A 
total of 280 responses, from 1 1 classrooms, revealed 
several interesting results. 

Satisfaction varies by course discipline. 

Satisfaction varies by instructional technology used in 
the classroom. 

There is no significant interaction of discipline and 
technology. 

Gender affects satisfaction. Females were less 
satisfied than males. 

Technology usage rates affect satisfaction generally; 
more usage equals higher satisfaction. 
Anticipatedgrade affects satisfaction modestly; lower 
grade equals lower satisfaction. 

Factor analysis did not reveal the five theoretical 
dimensions of SERVQUAL The researchers found at 
most four factors; in these the Empathy and Assurance 
dimensions were generally commingled. The 
Tangibles dimension could n ot generally be identified. 

The 20 main service satisfaction items were unchanged 
from 1999 to 2001. The current study used improved 
instructions for the classroom administration. This 
aspect was successful. The researchers used a less time 
consuming, but less personal approach to soliciting 
participants, and the smaller number of responses 
suggests the sampling methodology was not as 
successful. 

The researchers used improved statistical techniques 
with the2001 data, so the exact time series comparisons 
were not made. In the 1999 results, male respondents 
were less satisfied than female respondents; in 2001 this 



finding is reversed. In 1 999, economics students tended 
to be least satisfied; in 2001, that result fell to AIS. 
Otherwise, the basic nature of the two sets of results is 
consistent. 

To remove the impact of a relatively small number of 
classes influencing findings, the study should be 
repeated with large numbers of students in each course 
discipline and technology group. 
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ABSTRACT 

Academ ic dishonesty among college students appears to be a serious problem at many institutions . The problem may 
get worse in the future, as the availability of computer technology can make it even easier to cheat than in the past. 
Due to the nature of compu ter-based assignm ents used in information systems (IS) courses, the IS instructor is faced 
with the challenge of managing cheating among students. The purpose of this manuscript is to introduce a risk 
assessment framework to help IS instructors develop appropriate controls to address cheating. The framework 
recommends that four types of controls beused to address cheating: deterrent, preventative, corrective and detective 
controls. It is suggested that detective controls are especially critical. To illustrate the ideas discussed in the 
manuscript, an example of an approach to address cheating in an IS course is described. Experiences with the 
approach are discussed. 



INTRODUCTION 

Cheating among college students appears to be a 
problem at many institutions. A recent book by Cizek 
(1999) summarizes 16 studies on college cheating since 
1970, and concludes that the percentage of studentswho 
admit to cheating is “remarkably and uniformly high” 
(Cizek, 1999, p. 21). The studies consistently reveal that 
over 50 percent of the students admit to cheating on 
exams or other assessment methods. College cheating is 
not a new ph enom enon — an early experimental study of 
126 students in a women’s college by Drake (1941) 
found a cheating rate of 23 percent. However, the 
proportion of college students who cheat appears to be 
increasing. A review of the literature by Cole and 
McCabe (1996) found that the number of students who 
have admitted to cheating on exams has been rising over 
the past three decades. A recent study that surveyed 
more than 6000 college students at 31 campuses found 
that 70 percent of students surveyed admitted to one or 
more instances of cheating on an exam (McCabe & 
Trevino, 1996). For college instructors who teach in the 



area of business, the findings related to student cheating 
are even more disturbing. According to McCabe and 
Trevino (1995), academic dishonesty in undergraduate 
business programs may be even worse than other areas 
of study, as undergraduate students aspiringto careers in 
business self-reported a greater frequency of cheating 
than their peers in the areas of engineering/science and 
social science/humanities. These results suggest the 
emergence of a culture within business schools where 
some types of cheating are considered acceptable. 

As if to add fuel to the fire, the increased use of 
informationtechnology (IT)in business education offers 
the potential to make it easier to cheat. Reports in the 
popular media have highlighted how the Internet and 
other technologies present new opportunities for 
cheating (e.g., Kleiner & Lord, 1999). For example, 
plagiarism is one form of cheating that may be easily 
supported with technology. WWW sites sporting names 
like "fastpapers.com" offer simple tools to download 
prewritten term papers at the click of a mouse. It 
appears that students are taking advantage of such tools, 
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as these types of sites boast of more than 1 million 
downloads of term papers since their inception (Clayton, 
1999). In this manuscript, we refer to cheating that is 
facilitated by information technology as “E-Cheating.” 
While tools have been developed to help combat 
problems associated with E-C heating(Gilgoff, 2001; Joy 
& Luck, 1999; Ryan, 1 998; Whale, 1 990), the potential 
for E-cheating remains strong. 

The trends cited above have important implications for 
those who teach in Information Systems (IS) programs 
within business schools. Many IS curriculums rely 
heavily on electronically based assignments, projects, 
and exams. Cheating can be a sim pie act to perform in 
computer skills courses since a student can simply copy 
(or cut and paste) portions of the software from a 
classmate or another source into his or her own 
submission deliverable. While it can be easy for a 
student to E-Cheat, it can be very difficult for a teacher 
to tell the difference between honest work and 
plagiarized work. For instance, introductory computer 
skills courses typically require all students to address the 
same problem using the same data and the same problem 
specifications. Assignments that require a student to 
create specific formula features for a spreadsheet or 
specific tables and queries for a database are likely to 
result in very similar solutions across students. This 
presents a situation in which it is very difficult for an 
instructor to detect — or prove — that cheating has 
occurred, as the solutions that the students create should 
be very similar. 

What can an IS instructor do to address E-Cheating? 
The purpose of this manuscript is to introduce a “risk 
assessment” framework to assist in the development of 
E-Cheating controls that may be applied to IS courses. 
The paper is organized into five sections that l)review 
the literature on cheating and summarize the variables 
that are related to increased incidents of cheating, 2) 
introduce the electronic environment associated with 
many IS programs and the new vulnerabilities resulting 
from this environment, 3) propose a framework that can 
be used to design andevaluate a setof controls aimed at 
influencingthe vulnerabilities^) provide an example of 
a control environment developed using the proposed 
framework, and 5) summarize our experiences using 
these controls. 

VARIABLES ASSOCIATED WITH CHEATING 

One of the first steps in developing controls to reduce the 
incidence of cheating is determining the factors that 



influence, or are related to, incidents of cheating. Cizek 
(1999) reports that prior research has identified more 
than 100 variables that are associated with cheating on 
exams. It must be noted that these variables are not 
necessarily “determinants” and should not be construed 
as the direct cause of cheating. Rather, these variables 
are associated with an academic environment that 
influences the likelihood of cheating. 

Cizek classifies these variables as demographic 
characteristics, psychological constructs (e.g., 
personality traits, behavioral dispositions, etc.), and 
classroom instructional and environmental factors. 
Variables that have been studied (andthere correlations 
with cheating behavior) include the following. 

Demographic Characteristics 

• Cheating has been inversely related to 

achievement — students with lower grades are more 
likely to engage in cheating activity. 

• Younger students are more likely to engage in 
cheating activity than older, non-traditional students. 

• Cheating activity is more common among students 
who are members of Greek social organizations 
(fraternities and sororities). 

• There are no significant differences in cheating 
activity associated with gender. 

• There are no significantrelationships between cheating 
activity and identification with a religion. 

Psychological Constructs 

• Strong relationships have been reported between 
cheating and learning-grade orientation. Specifically, 
students who are motivated to earn a grade (as 
opposed to learn) are more like ly to engage in cheating 
activities, 

• Other variables that are significantly related to 
increased cheating behavior include: lenient student 
attitude about cheating, lower perceived riskof being 
caught, and the degree to which the student believes 
cheating is justified. 

• While a number of other psychological-oriented 
variables have been investigated in the literature (e.g., 
achievement motivation, personality type), the 
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research findings do not suggest that these variab les are 
strongly related to cheating behavior. 

Classroom Instructional and Environmental 
Factors 

• The following variables have all been associated with 
increased cheating activities: larger class size, free 
seating, poor supervision of the exam, inexperienced 
faculty, provision ofan opportunity to cheat, and take- 
home exams. 

• The following variables appear to reduce or deter 
cheating activities: warning about the penalties for 
cheating immediately before an exam, using different 
exams with differing questions and answer choices, 
using essay exams, student perceptions that the exam 
is fair and maintaining high instructor vigilance during 
the exam. 

• There does not appear to be any re lationship between 
cheating activities and the use of multiple exam forms 
where the same questions and answer choices are 
randomly positioned on different versionsofthe exam. 

It is interesting to note that an instructor can directly 
control the instructional and environm ental set of factors 
listed above. This suggests that instructors can take 
actions to control cheating in their courses. For 
example, in a survey conducted by McCabe and Trevino 
(1995), business students rated the chance of getting 
caughtandthe severity of cheating penalties as the major 
deterrent for cheating. Yet, it does not seem that 
instructors are effectively controlling the instructional 
and environmental factors, as student surveys reveal 
environments where faculty do not detect cheating, and 
where students who do cheat face trivial penalties 
(Franklyn-Stokes& Newstead, 1995; McCabe & Drinan, 
1999). While over 50 percent of the students surveyed 
admit to cheating, fewer than 2 percent ever admit to 
being caught (Haines, et al., 1986; Fishbein, 1994). 
Cizek (1999) concludes that the current high rates of 
cheating activity are unlikely to decrease because 
students perceive thatthe chance ofbeing caught is slim, 
and many believe that the chance of harsh pun ishmen t is 
minim al. 

Note that this lack of punishment is not due to the lack 
of institutional policies and procedures. Aaron’s (1 992) 
survey of higher education institutions indicated that 
over 95 percent had an ac ademic integrity code, and over 
98 percent had specific procedures for handling 



allegations of academic misconduct. However, only 7 
percent of the institutions reported that faculty mem bers 
addressed academic integrity policies in theirclassrooms 
or course syllabi. Thus, while policies and procedures 
are well represented in catalogs and handbooks, they are 
poorly represented in the environments where cheating 
occurs (i.e., classrooms). 

INFORMATION SYSTEMS COURSES: 

VULNERABILITIES TO E-CHEATING 

Since “hands on” assignments and projects provide an 
important means for assessing a student’s understanding 
of software development (e.g., spreadsheet development, 
visual programming), many IS instructors rely heavily 
on electronically based assignments and projects as 
evaluation tools. Based on our observations and 
discussions with faculty, many instructors allow their 
students to complete these electronically-based 
assignments in uncontrolled, unconstrained and 
unsupervised environments (e.g., the students are given 
the assignment an d then go off and do it). E-c heating is 
not difficult to do in such environments. In some cases, 
IS instructors will also rely on “hands on” types of 
assignments for exams that are provided on-line to the 
students. While such exams are typically given in a 
controlled environment (e.g., a computer laboratory) 
under instructor supervision, there are still a number of 
simple approaches that students can employ to share 
electronic files among themselves (e.g., e-mail, copying 
files to common areas or remote web sites, etc.). Even 
in the more restrictive setting ofa test environment, high 
instructor vigilance may not detect the inappropriate 
exchange of information. 

Unfortunately, with regard to the issue of managing E- 
cheating, it is possible that things may get more 
problematic for some IS faculty before they get better. 
As with many areas of the university, IS faculty are 
starting to teach distance education courses or on-line 
versions of their courses. Since it can be more difficult 
to monitor students in the electronic learning 
environment, and it can be easy to cheat in an 
environmentinvolving electronic deliverables, distance- 
learning settings offer the potential for increased 
opportunities to cheat (Kennedy, et al. 2000). 
Additionally,most ofthe institutional academic integrity 
policies are outdated with respect to information 
technologies and thus often fail to provide helpful 
guidelines for students. For example, Kennedy, et al. 
(2000) report that there appears to be little guidance 
within institutional policies on the proper use of the 



238 



Proceedings of the 16 h Annual Conference of the International Academy for Information Management 





BEST COPY AVAILABLE 



Internet in academic endeavors. While it may be clear 
what constitutes a plagiarism case for a English writing 
assignment, it may not be clear to students whether it is 
acceptable to surf the web to find solutions for a 
programming assignment. 

RISK ASSESSMENT FRAMEWORK 

What can an IS instructor do to address the issue of E- 
cheating? One approach is to perform a risk assessment. 
A risk assessment requires the evaluation of threats, 
vulnerabilities and controls (i.e., countermeasures for 
vulnerabilities). The threat in this case is that cheating 
will occur. The vulnerabilities are factors that make a 
threatmore likely to occur. In the context of E-cheating, 
to help identify key vulnerabilities, we may draw from 
our earlier discussion of the variables identified in the 
literature that have been associated with increased 
cheating activity. Lastly, it is necessary to identify 
controls that may be used to reduce the threat of 
cheating. There are four general types of controls that 
may be used as countermeasures: 1) deterrent controls 
that reduce the likelihood of the threat, 2) preventative 
controls that protect against the vulnerabilities and 
reduce the incidents of cheating, 3) corrective controls 
that reduce the effect of cheating activities, and 4) 
detective controls that discover whether cheating has 
occurred. An effective control environment must have 
a combination of strategies that ensure that all four types 
of controls are provided. 

Developing controls around a single vulnerability (i.e., 
a variable associated with cheating) may not affect the 
likelihood of cheating — a blending of controls aimed at 
addressing multiple vulnerabilities is likely to be more 
effective. When considering the variables discussed 
earlier, one finds thatan instructor has less control over 
some variables, and more control over others. For 
example, the instructor cannot directly influence 
vulnerabilities resulting from the demographic 
characteristics of the students. Some of these variables 
may be influenced at the institutional level — for 
example, banning Greek social organizations on campus. 
However, most control environments do not consider 
demographic vulnerabilities. Some ofthe psychological 
constructs can be influenced with prevention strategies. 
For example, developing group norms in the class 
supporting honesty is a control directly aimed at 
influencing student attitudes towards cheating. Using 
reasonable, interesting, and fair exams isa control aimed 
toward influencing students’ rationalizations that 
cheating is justified. Also, offering a number of 



activities for students to demonstrate achievement is a 
way to reduce grade pressure. 

The majority of controls are aimed at influencing the 
classroom instructional and environmental factors since 
these types of variables are typically under the control of 
the instructor. Examples include using a larger facility 
forexams, maintaining high vigilance during the exam, 
and using different exams with differing questions and 
answer choices. 

Many studies have been conducted to investigate student 
perceptions of the effectiveness of various cheating 
prevention strategies for exams (Evans & Craig, 1990; 
Genereux & McLeod, 1 995; Hoi linger & Lanza-Kaduce, 
1996; McCable, et al., 1999). The followingprevention 
strategies have consistently been identified by students 
as strategies that may reduce the incidence of cheating 
on exams. 

• Develop group norms in the class supporting honesty, 
with stated policies and procedures and high 
punishment for getting caught(at both the institutional 
and course level). 

• Reduce grade pressure by offering a number of 
activities for students to demonstrate achievement. 

• Write reasonable, interesting, and fair exams. 
Numerous studies report student perceptions that 
exams using an essay format are more effective at 
preventing cheating. 

• Use an appropriate testing environment (e.g., a large 
classroom with students sitting at every-other-seat). 

• Maintain high instructor/proctor vigilance during the 
exam, 

• Use alternate forms of the exam. 

These suggested prevention strategies can be mapped 
into three of the four types of controls discussed above. 

Deterrent controls reduce the likelihood that cheating 
will occur. These controls include: 

• Institutional policies and procedures are intended to 
deter cheating — outlining the process and potential 
consequences if it is determined that academic mis- 
conduct has occurred; 
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• Developing group norms in the class supporting 
honesty is also intended to deter cheating activities; 

• Offering a number of activities for students to 
demonstrate achievement should deter cheating 
activities by reducing grade pressures; and 

• Finally, when exams are perceived by the students as 
being reasonable, interesting, and fair, cheating is less 
likely to occur. 

Preventative controls attempt to reduce the incidence of 
cheating. These controls include: 

• Using an appropriate testing environment, such as a 
large classroom with stude nts assigne d to every- other- 
seat. This reduces the opportunity for students to 
cheat; 

• Maintaininghigh instructorvigilance during the exam. 
While it can be argued that vigilance can detect some 
forms of cheating, the credibility of observational 
evidence relies on human interpretation of behavior 
and events. Research indicates that human 
observations are notoriously fallible (Loftus, 1979). 
Thus, some other form of substantiating evidence 
(such as a student admissio n) is usually ne eded to fully 
pursue the consequences for academic misconduct. 
Thus, we consider instructor vigilance as a 
preventative control, not a detective control; and 

• Numerous studies report student perceptions that 
exams using an essay forniat are more effective at 
preventing cheating. Thus, use of an essay format 
would be a preventative control. 

Corrective controls reduce the effect of cheating. 
Using alternate forms of the exam, with differing 
questions and answer choices, has this effect in that it is 
intended to reduce cheating. 

What is missing from these recommended prevention 
strategies are detective controls. From the student’s 
perspective, this lack of detective controls in the control 
environment negates the effectiveness of the other 
controls. For instance, consider the case where a school 
has an academ ic integrity policy that clearly outlines the 
consequences of cheating. Such a policy may not 
necessarily provide a viable threat if the students sense 
that the instructor lacks the ability to detect cheating 
once it has occurred (and thereby lacks the ability to 
carry out the stated policies and enforce the 
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consequences). We believe that the lack of detective 
controls contribute significantly to the reported high 
incidence of cheating on college campuses. 

If students perceive that there is a very low chance of 
being caught, then they are m ore likely to cheat. Within 
an electronic environment, this issue is exacerbated, as 
detective controls appear to be even more difficult to 
implement. 

EFFECTIVE CONTROLS WITHIN 
AN ELECTRONIC ENVIRONMENT 

In this section, we will describe a set of controls 
designedto deter E-cheating on IS-orientedassignm ents 
and exams using the risk assessment framework. To 
address the issue of E-cheating, we have implemented 
controls in each of the four areas discussed above. At 
the deterrent level of control, ourinstitution provides an 
elaborate set of policies regarding academic integrity, 
along with specificprocedures in the eventthat academ ic 
misconduct occurs. This includes disciplinary coun- 
seling for those students who have been found guilty of 
misconduct. These institutional policies and procedures 
are summarized in the syllabi of all IS courses. 

The first upper-division course within our IS curriculum 
incorporates a “desktop” application development 
project using Microsoft Access. Since all students 
complete the same, highly structured take-home 
assignments, the detectionof unauthorized collaboration 
and copying is difficult. Since we cannot control this 
vulnerability, we have reduced the weights of these 
assignments towards the final grade and encouraged 
cooperative learning among the students. Thus, we are 
using a deterrent control to reduce the grade-orientation 
on assignments and place a higher focus on learning. 
Recall that a learning orientation has been associated 
with reduced incidents of cheating. 

To evaluate the student’s development of appropriate 
development skills, we have created an evaluation 
assignment thatrequires the student to develop adesktop 
application in Access. The evaluation assignment is 
comprehensive and simulates converting an index-file 
management system to a normalized database 
application. The students are given an Excel spreadsheet 
which contains data from the indexed-file system in a 
first normal form (FNF) format. The spreadsheet is 
imported into Access, and m ake-table queries are used to 
split the FNF table into third normal form (TNF) tables. 
The TNF tables must be modified to meet data 
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specifications (data type, length, format, input masks, 
etc.) and documentation standards (captions and 
descriptions). Once primary and foreign keys are set, 
referential integrity constraints must be established 
between the TNF tables. The application specifications 
require development of queries, forms and a main 
switchboard. While other development aspects are 
covered in the class (reports, customized menus, etc.), 
they are not included in the evaluation due to time 
constraints. 

As a preventative control, the students complete the 
evaluation assignment in a controlled environment. 
Specifically, the students are required to take the 
evaluation assignment in our computing laboratory, 
under supervision, within a specified time constraint. 
We make it very clear at the start of the evaluation that 
this is not a cooperative learning assignment. We also 
explicitly state, both verbally andin writing on the exam 
instructions, that e-mail and web activity may be 
monitored during the exam. Our institution does have 
privacy policies that prevent normal inspection of 
student e-mail and private file areas. When cheating is 
suspected, and other evidence suggests that electronic 
means were used in the cheating incident, we are able to 
obtain permission from the University’s Legal Office to 
inspect those student accounts during the specific time 
period of the exam. We consider this “threat of 
detection” as a deterrent control (it is not a detective 
control by itself since other evidence of cheating must be 
provided before its use is authorized). 

For a corrective control, we add complexity in a time- 
constrainedenvironment. Each student is provided with 
a different data set, and multiple assignment formats are 
systematically assigned to the students (rather than 
randomly assign formats, we ensure that each memberof 
a project team receives a different format). This 
discourages the exchange of formulas and methods via 
e-mail or other electronic means. 

Our approach for detection is to provide each student 
with a unique, identifiable data set. These data sets are 
distributed just prior to the on-line evaluation. At the 
end of the designated time, the completed assignments 
are check for the unique identifier and matched to the 
student. We are also able to check the author and 
creation time properties of the submitted file. With this 
detection scheme, we can detect if students shared files, 
and can determ ine the spec ific indi vidu als who colluded. 



OBSERVATIONS 

We are in our second year of using this control 
environment in the junior-year course. Prior to 
implementing this approach, we heard complaints from 
some students that some of their classmates were 
cheating on the electronic evaluation assignments. 
However, we were unable to verify the complaints of 
cheating since we did not have an effective detection 
scheme. Since implementing this approach, we believe 
thatthe incidence ofcheating has dropped to a negligible 
level. We no longer hear complaints about cheating. In 
the first term that we applied these procedures in the 
junior-level course, we detected four cases of attempted 
cheating on the evaluation assignment (out of 90 
students). In each of these instances, we were able to 
identify the individual who copied, and the individual 
who knowingly shared their solutions. All students were 
subjected to the appropriate disciplinary action, 
following the established institutional procedures. 
Although we did not publicize the cheating incidents that 
were detected, the word got out to otherstudents thatthe 
detection approach was effective. Since then, there have 
been no detectable instances of cheating on the 
evaluation assignment. 

Based on these observations, we conclude that the 
combination of controls is an effective deterrent for 
cheating during on-line examinations. Nonetheless, 
enterprising students will still find ways to cheat. For 
example, we did discover one instance where a student 
had multiple logins during the evaluation. Identifying 
who is logged into the system is a relatively easy task in 
most network environments, and we have now added 
this check as a control during our on-line exams. As 
with any control system, it is difficult to create 
manageable systems that cannot be defeated. Our goal 
has been to reduce the likelihood of cheating by reducing 
the opportunity to cheat. At this point, we believe that 
it takes more knowledge and effort to defeat the co ntrols 
than to complete the evaluation assignme nt. 

CONCLUSION 

In this article, we have reviewed a number of the factors 
that may be associated with cheating and have provided 
a framework that may be used to address E-cheating. In 
our view, detection — alongwith subsequentenforcement 
of institutional policies and penalties — are critical 
components for the effective control of E-cheating in IS 
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courses (as well as other courses). Many existing 
controls lack these two vital elements — resulting in the 
epidemic levels of cheating on campuses. The approach 
that we have developed to address E-cheating associated 
with our IS evaluation assignments appear to be 
working. We acknowledge that our controls are not 100 
percent effective. However, our experience offers 
encouragement that there are ways to combat E-cheating 
in the IS classroom. 
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' ABSTRACT 

Given the continued importance of training for information technology (IT) professionals, the ability to carefully 
manage training interventions is critical. One of the commonly recognized antecedents to successjul training, trainee 
motivation to learn (Holton, 1996), appears to have had a limited conception in past IT research (Sein, Olfman and 
Bostrom, 1987 ; Venkatesh, 1999). While IT research has considered it to be a unidimensional construct, research 
in other domains indicates that it may possess multiple dimensions. This paper examines the dimensions of the 
motivation to learn construct in multiple domains, develops a training process research model that focuses on the 
motivation to learn construct, describes possible future research based on that model, and then provides implications 
for IT educators and trainers . 



INTRODUCTION 

Technology innovation is constantly and pervasively 
altering the way in which work is done. In order to 
maintain position or exceed current norms, information 
technology (IT) companies are constantly recalibrating 
competencies as new technologies and updates of 
existing technologies are released. To exploit and 
advance technologies, organizations must either retrain 
existing IT professionals or hire new IT professionals 
(Ryan, 1999). Given this environment, IT workers 
pursue their vocation under the auspice of innovation, 
which mandates continual learning. 



Students aspiring to enter or advance their careers in the 
IT domain encounter a profession mandating lifelong 
learning. Learning may occur through various means 
including direct experience, vicarious experience, self- 
directed exploration ofweb or book content, and formal 
instruction through either educational institutions or 
training sources. In providing formal instruction to the 
intended audience, educational and training institutions 
may have to overcome the challenges of distance, 
efficiency, and access. Various forms and applications 
of technology have been introduced into the instructional 
environment to meet these and other challenges. Indeed, 
the use of instructional technology has boomed both in 
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educational and training contexts (Alavi, 2001). The 
current landscape is such that IT learners will likely 
encounter formal instruction in a technology-mediated 
format in their pursuit of domain knowledge. 

Unfortunately, even with best efforts and an extensive 
investment of resources, training and education is 
effective for some and not for others. Hence, it is no 
surprise that IT training frameworks integrate cognitive 
psychology, educational psychology, information 
science, and computer science. For example, a 
framework proposed by Bostrom “...indicates that the 
target system, training method, and individual 
differences, individually and in combination, affect a 
user’s mental model to produce training outcomes, 
which include learning performance and attitudestoward 
using the target system on the job (Bostro m, 01 fman , & 
Sein, 1990). 

One means to enhance IT training effectiveness with 
respectto individual differences is to customize training 
to knowledge gaps (Gjest land, 2000). Another approach 
involves moderating trainee learning states (trainability) 
that may impede training effectiveness or alternatively, 
facilitating cognitive and attitudinal factors that enhance 
trainingeffectiveness, A combined approach attempts to 
accommodate supportive trainee learning states as well 
as knowledge gaps through training customization. 
Skilled educators delivering traditional, stand-up 
instruction may use a combined approach as they 
develop lesson plans and deliver instruction to suit both 
perceivedknowledge/skill gaps and the learning state of 
students. 

Technology-mediated delivery mediums for learning 
introduce substantial challenge to delivering a combined 
approach that addressesknowle dge/skill gaps as well as 
learner attitudes and cognitive states. How can we 
incorporate into computer based trainingprograms what 
has often come from “intuition, common sense, and trial 
and error (Wlodkowski, 1985)? Nevertheless, in the 
quest to address issues associated with distance, 
efficiency, and access, technology-mediated learning 
(TML) may have to face this hurdle to enhance and 
improve (or even to replicate) traditional learning 
effectiveness. In a recent article, Alavi asserts not only 
a call for greater depth and breadth of research into 
technology-mediated learning, but highlight the void in 
research studies that .focus on forming relationships 
among technology and relevant instructional, 
psychological, and environmental factors that will 
enhance learning outcomes (Alavi, 2001). 



Trainability is hypothesized to be a function of three 
trainee factors: ability, motivation and perceptions of 
work environment (Noe & Schmitt, 1986; Waxley & 
Latham, 1981). Researchers have found a positive 
relationship between motivation and training outcomes 
(Baldwin, Magjuka, & Loher, 1 99 1 ; Fracteau, Dobbins, 
Russell, Ladd, & Kudisch, 1995; Maier, 197 3; Warr & 
Bunce, 1995). Vroom (1964) suggested that when 
motivation is low, both high and low ability individuals 
exhibited low levels of performance and when 
motivation is high, ability has a greater influence on 
performance. “Moti vation is not only important because 
it is a necessary causal factor of learning, but because it 
mediates learning and is a consequence of learning as 
well (Wlodkowski, 1985). Unfortunately, the moti- 
vation construct brings with it a cornucopia of 
assumptions, terminology, and dimens ions, particularly 
regardingadult learner motivation (Wlodkowski, 1985), 
A better understanding of the motivation to learn 
construct should facilitateinterventions that can enhance 
positive training outcomes and return on investment 
(Baldwin, Magjuka and Loher, 1991 ; Fracteau, Dobbins, 
Russell, Ladd and Kudisch, 1995; Warr and Bunce, 
1995). This paper proposes a research model that 
attempts to “specify the domain of the motivation to 
learn construct as one means offortifyingthe foundation 
of IT technology-mediated learning research. 

PRE-INSTRUCTION MOTIVATION 

Participants enter and leave training with 
varying levels of motivation that will likely 
influence how much they learn, whether they 
transfer learning to the job, and ultimately how 
successful the program is (Mathieu & 
Martineau, 1997). 

Motivation is generally thought ofas more of a state than 
a trait. As such, motivation is subject to malleability 
(Wlodkowski, 1985); it is possible that educators can 
positively affect a student’s state of motivation to 
enhance the potential for effective training and 
education. Motivation measures can be used both pre- 
and post-training to assess the trainee’s level of 
motivation (Kirkpatrick, Craig, & Bittel, 1996). 

Students with a high pre-instruction motivation can be 
described as being “primed for learning, meaning they 
would be more attentive to learning training material and 
amenable to new ideas (Mathieu,Tannenbaum, & Salas, 
1992). It is not surprising that most training studies 
recognize a high motivation to learn as an antecedent to 
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learning and ultimately to training effectiveness 
(Baldwin et al., 1991; Goldstein, 1993; Holton, 1996; 
Mathieu & Martineau, 1997; Ryman, 1975; Sanders & 
Yanouzas, 1983; Tannenbaum, Mathieu, Salas, & 
Canon-Bowers, 1991). Correspondingly, researchers 
have found people with a low motivation to learn 
generally do not achieve desired training outcomes 
(Maier, 1973). Clark, Dobbins and Ladd (1993) assert, 
“...without motivation to learn, the most sophisticated 
training program cannot be effective. 

Measuring pre-instruction motivation as a means for 
assessing customization needs may be a suitable and 
relevant initial step in customizing technology-mediated^ 
learning (TML) to student motivational needs. In tradi- 
tional forms of instruction, the educator may employ 
obtrusive forms of motivation assessment in union with 
or independent of unobtrusive forms either before the 
instruction begins as part of the instructional design 
process or as part of opening the instructional process to 
determine an appropriate path for the instruction. An 
instructor or instructional designer may execute 
obtrusive forms of measurement such as a written or 
verbal survey to measure the student’s motivational 
state. Little, if any, literature exists indicating obtrusive 
means of pre-training motivation to learn would 
compromise the instructional process. However, once 
the actual instructional process begins, it is doubtful the 
instructor would continue to use obtrusive forms of 
measurement as such measures may cause distraction 
and interrupt the flow of the instructional process. If an 
instructor had reason to reassess motivation once the 
instructional process began, they may more likely rely 
on unobtrusive measures, particularly observing student 
behaviors (Wlodkowski, 1985). 

Incorporating appropriate obtrusive motivation to learn 
measures, such as validated survey instruments, into 
many TML medium s is well with in the capacity of many 
existing technologies with minimal associated costs. 
Alternatively, using an instrument, apart from the TML 
medium (e.g. pen and paper or verbal) is possible. 
Measures of pre-instruction motivation may determine 
the appropriate customized “tone of the TML (such as 
the need for recurring positive feedback within the 
program) or even indicate the TML would not be 
appropriate given the learner’s current motivational 
state. Mathieu, Tannenbaum and Salas (1992) contend 
trainees without pre-training motivation are unlikely to 
achievelearning objectives, even if they find thetraining 
program enjoyable. Hence, such a measure provides 



important information pertinent for customization 
direction even in the presence of an enjoyable TML 
session. 

Conversely, though measures of motivation during the 
course of a TML session would provide valuable 
information in tandem with pre-training measures for 
adaptation purposes, unobtrusively acquiring such 
information may prove more difficult in the current 
environment and require a somewhat different set of 
measures. Measures of motivation during training may 
seek to assess the learner’s current state of interest, 
enjoyment, and arousal in response to the stimulus of 
learningcontent and delivery style. Affectiveassessment 
software based on bio-sensory and/or physical 
appearance readings may provide welcomed promise as 
means to measure motivation without introducing an 
instrument that may disrupt the learning process. 
However, such technology seems to still be in 
development stages (Lisetti, Douglas, & LeRouge, 
2001). Even in applying such tools to the TML context, 
a pre-training motivation assessment still holds as a 
critical first step in facilitatingthe learning process. By 
assessing pre-training motivation, stakeholders can 
diagnose if trainees are ready to accept training 
(Goldstein, 1993) and potentially introduce facilitating 
interventionsinmotivationdeficientsituations before the 
training process actually begins. 

DeLone and McLean (2001) urge researchers to pay 
careful attention to context in defining and measuring 
each component of success included in their information 
systemssuccess model and assessing inter-relationships 
between success constructs. Research also indicates that 
users prefer to tailor success constructs and measures to 
the type of system under evaluation to facilitate 
understanding and application (Jiang & Klein, 1999). 
Correspondingly, Alavi notes that the IS discipline itself 
“...may influence appropriate uses of TML (Alavi, 
2001). Therefore, decomposition of a potentially multi- 
dimensional construct, like pre-instruction motivation, 
under the auspice of the domain characteristics and 
boundaries present in IT TML will provide ahigh degree 
of relevance and understanding to researchers and 
practitioners. 

Decompositionofthis constructprovides researchers and 
practitioners with a connecting framework to facilitate 
TML success by considering the extent, nature, and 
appropriateness for various types of motivation 
interventions or customization in each situation of use. 
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Without an understanding of pre-instruction motivation 
in the IT TML environment, the potential for successfijl 
implementation of IT TML is diminished. 

For researchers to study pre-instruction motivation in the 
context of IT technology-mediated learning, they must 
“specify the domain of the construct to the extent of 
exactness possible to define the conceptual contentofthe 
construct (Churchill, 1979). Without such study, it is 
questionable whether instruments applied or used would 
possess desirable psychometric properties and produce 
reliable results (Churchill, 1979). Though many studies 
and conceptual models positionmotivation to learn as an 
important construct in training and educational settings, 
comparing and synthesizingstudies for potential transfer 
into the IT TML domain is difficult since the construct 
definition and dimensionality have varied across studies 
and have produced some mixed results (Baldwin et al., 
1991; Mathieu et al., 1992; Noe & Schmitt, 1986). 

The purpose of this study is to address that need by 
developing a research model for IT technology-mediated 



learning that encompasses the multiple dimensions of the 
motivation to learn constructthat have been proposedby 
past research in various domains for testing in the IT 
TML domain . Future research based on this model will 
be proposed and implications for educators and trainers 
will be discussed. 

RESEARCH MODEL 

The fields of education, management, and psychology 
are the primary sources of prior research on the 
motivation to learn construct (Kanfer & Ackerman, 
1989; Mathieu et al., 1992), although a limitedbody of 
work can also be found in the IS and instructional 
technology domains (Igbaria, Parasuraman, & Baroudi, 
1996; Sein, 1987). Each domain has presented its own 
version of the motivation to learn construct; construct 
definitions from each domain are given in Table 1. 

The construct's commonality between fields isthat each 
field recognizes this motivation construct as a factor in 
training effectiveness. The various representations of the 



TABLE 1 

PRE-INSTRUCTION MOTIVATION CONSTRUCT AS REPRESENTED IN MULTIPLE DOMAINS 



Literature Domain 


Construct Definition 


Education 


“[T]he meaningfulness, value, and benefits of academic tasks to the learner 
regardle ss of whether or not they are intrinsically interesting (Marshall, 
1987). Motivation to learn has also been characterized by long-term, 
quality involvement in learning and commitment to the process of learning 
(Ames, 1990). 


Management 


“[T]he extent to which trainees are stimulated to attend training, learn from 
training, and use the skills and/or knowledge obtained in training after 
returning to the job (M athieu & M artineau, 1 997). 


Psychology 


The “force that influences enthusiasm about the program (energizer); a 
stimulus that directs participants to learn and attempt to masterthe content of 
the program (director); and a force that influences the use of newly acquired 
knowledge and skills even in the presence of criticism and lack of 
reinforcement for use of the training content (m aintenance) (Noe & Schmitt, 
1986). Also, the proportion of the subject’s total capacity devoted to the task 
(Kanfer & Ackerman, 198 9). 


Instructional 

Technology 


Motivation to learn determines the magnitude and direction of learner 
behavior and is intertwined with the effectiveness of learner performance. 
Motivation to learn must be considered in all parts of an instructional 
message and is affected more profoundly by factors other than 
entertainment (Keller & Burkman, 1993). 


Information Systems 


Sein, Bostrom and Olfman (1987) associated motivation to the desire to use 
the technology skills conveyed in the training process. Venkatesh (1999) 
furthered this link to technology by tying perceived usefulness to the 
technology. 
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construct also seem to all suggest that the effects of pre- 
training motivation to learn seem to extend beyond the 
actual training experience to the ultimate objective, the 
transfer of skills acquired in training to the job context 
(Mathieu & Martineau, 1997). Representation of this 
theory can be found in the IT context; Ryan (1999) 
theorized pre-training motivation to be part of the 
learning process that may affect training effectiveness 
for the IT professional. 

Domain perspectives vary in that researchers deviate in 
their attempts to explain motivation in terms of 
attitudes, beliefs, values, expectancies, attributions, 
needs, motives, deprivations and incentives, or 
reinforcements (Keller & Burkman, 1993). The 
disciplines also differ in the labels used to describe pre- 
instruction motivation. This incongruent representation 
presents a significant challenge to an in-depth 
understanding of this construct both in research and in 
practice. 

Many studies presentpre-instruction motivation as auni- 
dimensional construct. However, the measures used and 
conceptualization of the construct among the various 
studies does not converge. Generally, past studies that 
examine pre-instruction motivationas auni-dimensional 
construct only reported Cronbach’s a as a means to 
validate internal validity for the measure (Mathieu et al,, 
1992; Noe & Schmitt, 1986). Though Cronbach’s a is 
high in a number of these studies, Cronbach’s a may 
provide incorrect results when the measure is 
multidimensional (Cronbach, 1951). This inaccuracy 
could plausibly lead to incorrect research conclusions. 

This variation in definitions may suggest that pre- 
instruction motivatio n is a multi-dim ensional co nstruct. 
This has direct implications in the IT TML domain as it 
suggests that prescriptive interventions, such as TML 
customization, to address motivation deficiencies may 
vary according to the specific type of motivational 
deficiency. Some studies do investigate pre-instruction 
motivation as a multi-dimensional construct. The 
schema presented in Figure 1 is a decomposition ofthe 
dimensions of the pre-instruction motivation construct. 
This decomposition was derived through an informal 
meta-analysis of significant works (noted on Figure 1) 
indicating multi-dimensional aspects of motivation to 
learn. Just as the term for pre-training motivation is not 
exact in each supporting study, the labels associated with 
some underlying dimensions of intrinsic motivation may 
also vary and cause confusing overlaps. The model 
aggregates literature based upon the conceptual meaning 
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of each construct conveyed in the study rather than upon 
the term used to label the construct. Discussion of the 
links and disjoins within this body of literature follow. 

The proposed research model (Figure 1) first segregates 
pre-instruction motivation into intrinsic motivation and 
extrinsic motivation. Lepper (1988) called pre- 
instruction motivation “motivation to learn and 
presented motivation to learn as a multi-dimensional 
construct, encompassing both intrinsic motivation and 
extrinsic motivation. Extrinsic motivation is rooted in 
behavioral psychology; learner performance or 
participation is associated with attaining a reward or 
avoiding punishment that is external to the learning 
activity. However, rewards given without links to 
performance reduce interest (Bandura & Schunk, 1981). 
Extrinsic motivation does seem to reference the IT 
domain. For example, Venkatash (1999) labeled 
perceived technology usage as an outcome expectancy 
and, hence, a uni-dimensional measure of extrinsic 
motivation. 

Though extrinsic motivation may have application 
relevance to the IT TML domain, the effectiveness of 
extrinsic reward structures on learning motivation is 
questionable (Ashton & Webb, 1986). Intrinsic 
motivation seems to have a stronger impact on 
instructional effectiveness. 

Studies indicate that intrinsically motivated learners 
employ richer mental models, information gathering 
strategies, decision-making strategies, and information 
integrationthan learners who were externally motivated 
(Lepper, 1988; Lumsden, 1994; Condry & Chambers, 
1978). Intrinsicmotivation may have greater relevance 
in developing customized TML that seeks to enrich 
cognitive process by engaging the learner to undertake 
a learning task “for its own sake or to enhance self- 
efficacy according to that user’s cognitive and affective 
state. A large body of research focuses on intrinsic 
motivation to learn. 

The model presented in Figure 1 indicates that intrinsic 
motivation consists of the following dimensions: 
enthusiasm (motivation for instruction), self-efficacy, 
and proximal motivation (achievement goals). Though 
there are many studies that reference one or two of these 
dimensions, few studies have concurrently emp loyed all 
three of these dimensions as separate and unique 
constructs in modeling motivation to learn. One 
exception is Mathe iu and Martineau’s (1 997) conceptual 
work that claims that intrinsic m otivation to learn is not 
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FIGURE 1 

DIMENSIONALITY OF PRE-INSTRUCTION MOTIVATION 
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one, but three concepts: motivation for training (shown 
as enthusiasm/motivation for instruction in Figure 1), 
self-efficacy, and valence-instrumentality-expectancy 
(shown as dimensions of proximal m otivation in Figure 
1). However, neither Matheiu and Martineau (1997), nor 
other researchers seem to have ope rationalized alfthree 
dimensions in one empirical study. 

In defining motivation for training, Matheiu and 
Martineau (1997) refer to the “extent that trainees are 
stimulatedto attend training, learn from training anduse 
the skills and/or knowledge obtained from training after 
returningto thejob. Hicks recogn ized this same type of 
constructas one measure of intrinsic motivation to learn, 
but referred to this construct as trainee enthusiasm 
(Hicks, 1983). Brophy makes a distinction between 
intrinsic motivation (e.g. learning for the sake of 
learning) and motivation to learn (referring to positive 
affective aspects of motivation, like enjoyment or 
enthusiasm) stating that intrinsic motivation may exist 
without the presence of motivation to learn (Brophy, 
1987). Brophy points out that intrinsic motivation 
(shown as self-efficacy and proximal motivation in 
Figure 1) may exist and induce performance where no 
motivation to learn exists (shown as enthusiasm in 
Figure 1) (Brophy, 1987). Conversely, motivation to 
learn can occur without enjoyment. Based on the 
foregoing discussion, it would appear that intrinsic 
motivation is composed of multiple dimensions, and that 
enthus iasm i s just on e of tho se dim ension s. 

Self-efficacy is anotherdimension ofintrinsic motivation 
in the learning context. In adapting Bandura’s definition 
of self-efficacy to atrainingcontext, self-efficacy can be 
thought of as a belief that one can master material with 
a reasonable amount of effort (Bandura, 1986). Studies 
examining learner self-efficacy indicate a positive 
relationship between self-efficacy and training 
performance and/or training reactions (Gist, Rosen, & 
Schwoerer, 1988; Gist, Schwoerer, & Rosen, 1989; 
Mathieu, Martineau, & Tannenbaum, 1993). However, 
these studies did not also examine enthusiasm and/or 
proximal motivation. In looking at motivation to learn 
within the IS domain, Sein,01fman and Bostrom (1987) 
recognized self-efficacy as self-concept(trainee’s view 
of ability to learn) and also recognized the application of 
goals in trainingthrough their discussion of the need for 
^ achievement and the desire to succeed at the learning 
task. 



Achievement goals relate to proximal motivation. 
Certain properties ofgoals such as specificity, level, and 
proximity enhance motivation. Proximity references 
how far into the future individuals are able to project 
goals, and how closely sub-goals relate distal goals to 
current performance (Ban dura & S chunk, 1981). Both of 
these dimensions affect self-motivational mechanisms. 
When learners can project the relationship between 
training and proximal goals or sub-goals, intrinsic 
motivation may be enhanced (Bandura & Sc hunk, 1981). 
In contrast, when instruction is associated with lofty 
distal goals, the large negative disparities between costs 
and benefits are likely to diminish intrinsic motivation 
(Bandura & Schunk, 1981). Kanfer reported that low- 
ability subjects benefit more from the imposition of a 
goal assignment declarative as part of the training than 
do high -ability subjects (Kanfer & Ackerman, 198 9). 

Researchers have usedelements of Vroom’s Expectancy 
theory (1964) as a theoretical underpinning for goal 
assignment (proximal motivation) by representing 
training performance as a joint function of cognitive 
ability and motivation (Kanfer & Ackerman, 
1989;Waxley & Latham, 1981). To recognize 
expectancy theory as a theoretical underpinning of 
proximal motivation, instrumentality, valance and 
expectancy have been added to the pre-training 
motivation model as dimensions of proximal motivation. 
Valence refers to the degree of positive or negative value 
the individual associates with a given outcome, based on 
cognitive anticipations of affect regarding future 
outcomes. Instrumentality reflects the individual’s 
assessment that a given performance level will lead to 
specific outcomes. Expectancy theory suggests that 
assuming a given level of ability and facilitating 
conditions, greater effort should lead to higher 
performance. 

Though the focus of this study is specifically on the 
motivation to learn construct (highlighted in Figure 2), 
a more comprehensive training model that depicts other 
relationships suggested from past research provides 
perspective. Motivation to learn is shown to have an 
influence throughout the training process, as presented 
by Steers and Porter (1975) who viewed motivation to 
learn as having energizing, directing and maintaining 
components. To exemplify these components, an ideal 
training situation would include: an enthusiastic trainee 
or educator, a skilled and stimulating training facilitator 
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(human or automated), and reinforcement and 
encouragement to apply newly acquired skills. 

FUTURE WORK 

The first step in testing whether intrinsic motivation to 
learn is a multi-dimensional construct as represented in 
the model would be to conduct survey research with 
measures for the various motivation to learn constructs. 
The data would be analyzed by an exploratory factor 
analysis followed by a confirmatory factoranalysis using 
different subject pools. It seems relevant to assess 
intrinsic motivation to learn using quantitative means 
since 1) studies indicate that intrinsic motivationto learn 
may have the greatest effect on long lasting outcomes 
(Condry & Chambers, 1978; Lepper, 1988) and 2) 
intrinsic motivation to learn seems to be particularly 



vulnerable to multiple forms of representation in the 
literature. Measures ofmotivation to learn thathave been 
frequently used in past training rese arch will be adapted 
for this study (Colquitt & Simering, 1998; Fracteau et 
al., 1995; Mathieu & Martineau, 1997; Noe & Schmitt, 
1 986). Given that IT novices will have to undergo much 
training and education during the initial stages of their 
career, subjects will be chosen that appropriately 
represent IT novices. 

Further research could explore the influence of the 
various dimensions of the motivation to learn construct 
on the process and outcomes of training and education. 
This would p otentially pro vide practic al prescriptio ns to 
compensate for low motivation situations, such as 
adjustments to training methods or learning processes 
that may find application in IT TML settings. 



FIGURE 2 

MOTIVATION TO LEARN - 
MULTIDIMENSIONALITY AND KEY RELATIONSHIPS 
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CONCLUSIONS 

From a research perspective, a better understanding of 
the motivation to learn construct may enhance 
representation of this construct in IT TML research. If 
motivation to learn is a multi-dimensional construct, 
measures used in research should be appropriately 
developed to ensure internal validity. Hence, research 
should collectively and separately explore these 
dimensions to reach appropriate conclusions. 
Additionally, a more precise and richerunderstanding of 
this construct can facilitate insight into comprehensive 
training process models and provide direction for 
customizing TML to a learner’s pre-training 
motivational state. 

From a practical perspective, the skills required of IT 
professionals are constantly changing and competitive 
intellectual assets may affect marketposition creating a 
constant demand for IT training and formal education 
programs. Instructional sources are turning to TML to 
meet someof these demands and address the challenges 
ofaccess, distance, and efficiency. The potential positive 
impact of effective IT training and education on 
businesses is substantial. Though significant invest- 
ments in TML are being made (e-leaming in the 
corporate sector alone is expected to grow to SI 4.5 
billion revenue in 2004 ), TML is not a panacea and tends 
to ignore what is known about learning, such as the 
benefits ofcustomizing instruction to meet learner needs 
and deficiencies (Munster, 2 001). 

Assessing pre-instruction motivation provides the 
opportunity to enhance the effectiveness of the learning 
experience (or mitigate the losses associated with an 
ineffective learning experience) by allowing 
stakeholders to respond proactively to the learner’s 
motivational state. Pre-training measures of the 
dimensions of motivation may target the appropriate 
TML customization or other form of pre-training 
intervention best suited to the learner’s motivation 
strengths and deficiencies. For example, if self-efficacy 
is found to be a significant dimension, educational 
studies indicate that a student in an environment that 
promotes self-worth, competence, and autonomy may 
more readily accept the risks inherent in learning. 
Hence, a TML environment that promotes enrichment 
and recurringpositive feedback may enhance motivation 
to learn for a trainee with low efficacy. 

To exemplify another condition, if an employee or 
student has a low proximal motivation measure, 



customization linking TML modules to career goals, 
interests, and skills may help increase motivation to 
learn (Noe & Schmitt, 1986). Alternatively (or in 
addition to TMLcustomization), the instructionalsource 
may choose to defer training until after the learner has 
received career guidance and mapping that associate 
training as a sub goal toward a specific distal or proximal 
goal. As an illustration, career workshops may provide 
students with a clearer picture of career paths and 
prerequisite skills forhorizontal and vertical movement 
throughout an organization. Such interventions may 
stimulate students’ exploratory behavior and motivation 
to learn (Noe & Schmitt, 1986), which may facilitate the 
benefit of TML. 

A better understanding of the motivation to learn 
construct may provide trainers and educators with the 
clarity required to more effectively and directly respond 
to challenges regarding motivation to learn issues with 
traineesand IT students. Targeting prescriptive practices 
to specific dimensions of the motivation to learn 
construct can potentially lead to a better return on 
training investments, an expansion of learning options, 
and increased satisfaction with the learning process. 
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ABSTRACT 

The Internet is a valuable too l in area of Finance . Since its inception , it has become a major source of information 
to the disciplin e . This paper is a description of a project at a college where finan cial concepts are com bined with 
a research component using the internet . Furthermore, an information management function is incorporated into 
the project. Another part of the paper deals with thefeedback on the project. Some the information available to the 
researchers is discussed and critiqued. Teamwork is another application used in this project. Overall, this project 
is deemed a success in that students had an opportunity to analyze real life scenarios. 



INTRODUCTION 

The impact of the Internet on the finance curriculum has 
been enormous. The purpose of this paper is to discuss 
the use of the Internet for a project in the Finance 
classroom. In addition, it will describe the information 
found on these sites. 

Web sites suchas Yahoo.com and CNBC.com have had 
a tremendous impact in the Finance. It is now possible 
to get quotes and even trade stocks over the Internet. 
Furthermore, many companies have web sites that 
provide annual report information. Some sites, like 
Wendy’s International, have it so that one may can 
download the information as a spreadsheet. 

In an increasingly technology-dependent economy, all 
students, not just finance students, need to be adept with 
Internettechnologies. Students must know about search 
engines, how to structure search queries, and how to 
maneuver through cyberspace. They must know how to 
follow links and download information. A basic 
knowledge of spreadsheets is also imperative to the 
success of each individual. 



One of the concerns of the professor involved is the 
simplicity of financial statements in textbooks. The 
professor believes real life examples would make for a 
better educational experience for the students. 
Consequently, this project was developed. 

Since businesses and theeconomy move quickly, the use 
ofthe most current data and the ability to rapidly analyze 
it is crucial to the business professional. One of the 
goals of the project is for students to identify potential 
sources of information to solve business problems and 
show how the internet can be used for that purpose. 

PREVIOUS RESEARCH 

No research specifically has been done on student 
projects such as this. However, Matsumoto and other 
surveyed professional financial analysts. They identified 
32 ratios that these professionals considered to be 
important in the analysis of companies. This study used 
some of these ratios as 13 are incorporated into the 
study. 
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BEST COPT AVAILABLE 



Two other studies identified financial uses of the 
internet. One offered a guide to how to do a search and 
the other identified some possible sources of 
information. 

PROJECT 

This project has evolved over the years. Initially, the 
students did these projects solo. However, a problem 
existed in that current industry averages were not 
currently available. Furthermore, Business Advisory 
Boards recommend group work since they expect their 
employees to work in groups to solve problems. 
Therefore, the use of groups was initiated in 1999. 
Teamwork provides foractive learning opportunities to 
occur. 

The course used for this study is the basic financial 
management course taught at the undergraduate level. 
Part of the course involves a project where the students 
have to do financial research on publicly traded 
companies. Students are paired in a group of two or 
three. Furthermore, students are allowed to select their 
teammates and companies. Each group picks a company 
per each student. The companies foreach group must be 
competitors of one another. For example, Exxon and 
Texaco would be good selections if the students are 
interested in energy companies. These companies must 
be publicly traded on an exchange so that the 
information is readily available. Companies must be 
approved in advance by the professor since banks and 
other financial institutions have alternative forms of 
financial statements in terms of presentation. Once a 
company was chosen, another team in the same section 
of the course could not use that company. 

The next step is for the students to obtain the necessary 
financial statements. Computations required are two 
liquidity ratios, current and quick. Liquidity ratios 
measure acompany’s ability pay its bills. Activity ratios 
include inventory turnover, day’s sales outstanding, and 
fixed asset turnover. Activity ratios measure how well 
the assets of the company are being managed. Leverage 
ratios include historical debt ratio, times interest ratio, 
and market based debt ratio. These ratios measure the 
use of debt by the company. Profitability ratios are gross 
profit margin, operating profit marg in, net profit margin, 
return on assets, and return on equity. These ratios look 
at the overall of the company in relation to the income 
earned. All required ratios are discussed at length in 
class. Three years of ratios are required to look at a 



possible trend. Three years of data is required. It 
provides for some research opportunities since most 
annual reports include only two years worth of balance 
sheets. 

After computations are made for their company, the 
students are required to compare these ratios between 
companies. For example, the liquidity of one company 
is compared to the liquidity of the competitor. 
Furthermore, they are to analyze any trends that exist in 
the data. Next, they have to decide whether they would 
invest in these companies. Anotherdecision to be made 
is whether to loan these companies money. Defense for 
their decisions must be based on the above information 
or any other relevant information. 

SOME OBSERVATIONS 

A survey was taken to determine where the students 
obtained their financial statements. With the exception 
of EDGAR and the other category, the faculty member 
showed how to navigate these web sites. The sources of 
information is as follows. 

TABLE 1 

SOURCES OF FINANCIAL INFORMATION 



Source 


Percent 


Yahoo 


50% 


Company web page 


35% 


CNBC 


‘ 12% 


EDGAR 


6% 


Other 


7% 



One observation related to the Yahoo web site is that 
sometimes it did not have the desired information in the 
research link. The information needed could be found 
on another link. EDGAR is not too user friendly since 
you have to scroll down to find the appropriate 
information needed. Some web sites were found to less 
than user-friendly. For example, some companies did 
not provide their financials or were hard to locate. The 
fact that some companies did not have financial 
information available on their website was surprising 
since websites have become an important source of 
information for investors. So me had data files that were 
downloadable in a spreadsheet format. Adobe Acrobat 
is the most popular medium on the website for 
presenting the financial information. The breakdown is 
as follows. 
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TABLE 2 

SUMMARY OF FINANCIAL INFORMATION 
AVAILABLE FROM COMPANY’S WEBSITE 



Format 


Percent 


Adobe 


34% 


No financial information available 


26% 


HTML 


22% 


Excel 


9% 


Edgar 


9% 



One of the major goals of the project is that students get 
to work with real life scenarios. Some students com- 
mented as to learning more about companies in working 
with them on the projects. It also exposed the student to 
financial statements that had anomalies that they had not 
been exposed to before. Unless they are accounting 
majors, assets such asgoodwill, capitalized research and 
development, deferred income taxes are accounts that 
students have not been exposed to. 

Other non-quantitative observations are that some 
students were not good team members. They did not 
perform up to their team member’s expectations. This is 
an occurrence when group work is used. As mentioned 
earlier, it represents an opportunity foractive learning to 
occur. A related issue at this school is the honor code in 
place. The wording of the honor code does not 
encourage group work by students. The professor, at 
times, had to encourage the students work together. At 
the present time, firing of team members for non- 
performance is not allowed. In the future, that may be 
allowed. 

FEEDBACK BY STUDENTS 

The following are comments from student evaluations of 
the course made by students for the last three years. 

1998 

1. Stock market project needs to be more specific or 
thrown out. 

2. I enjoyed how to read how well a corporation is 
doing by its ratios. 

3. Students need more explanation of expectations for 
paper. 

4. Financial analysis 



1999 

1. I thoroughly enjoyed the project. It helped me to 
betterunderstand the significan ce of financ ial ratios. 

2. In terms ofthe group project, that was not helpful to 
me and it was “busy work.” 

3. Learned things that could help me in the future 
about stocks and bonds. 

2000 

1. Helps me to understand other Business and 
Accounting Courses. 

2. Having the project due early in the semester was 
extremely appreciated. 

3. Interpreting financial ratios and statements. 

4. He was able to explain things well on the group 
projects. 

5. All aspects, it provided a good understanding ofthe 
principals of finance which is helpful at my job and 
evaluating company’s as investments. 

6. The project was interesting and showed how to 
apply the findings to determine the company’s 
strengths. 

7. The way weapplied the information learned in class 
to real life companies was very valuable. That 
aspect helped in realizing the importance of finance 
to organizations. 

8. Practical application project. 

9. The ratio project tremendously helped my grade. 

10. I found the project to be especially helpful because 
it allowed me to see how different companies 
compare and how the ratios are used to judge the 
financial stability of businesses. 

1 1. Analysis of financial statements. I feel I am better 
at understanding a company’s financial statement 
and its standing compared to similar companies. 

12. I think the use of actual companies for our examples 
in class and our projects was helpful in making us 
realize how what we are learning is used. 
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13.1 found the aspects of the course that dealt with 
stocks and realistic issues to be the most helpful. 

It is interesting that all of the comm ents from 200 0 were 
positive. Many more comments were made about the 
project in 2000. 
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APPENDIX A 

LIST OF RATIOS COMPUTED 



Current Ratio 
Quick Ratio 

Average Collection Period 
(Day’s Sales Outstanding) 
Inventory Turnover 
Fixed Asset Turnover 
Total Asset Turnover 



Debt Ratio 

Times Interest Earned 
A Market Based Debt Ratio 
Gross Profit Margin 
Operating Profit M argin 
Net Profit M argin 
Return on Equity 



APPENDIX B 

LIST OF COMPANIES USED BY STUDENTS 



Adidas 

Alltel 

Anheuser-Busch 

Cisco 

Compaq 

Coors 

Direct TV 

Ford 

Fuji 

Gap 

Gateway 

General Motors 

The Home Depot 

Kmart 

Kodak 

The Limited 



Lowe’s 

Lucent Technologies 

Nike 

Nortel 

Papa John’s 

Pizza Inn 

Reebok 

Sciquest 

Southwest Airlines 

Sprint 

Target 

Time Warner 
USAirways 
Ventro 
Wal-Mart 



258 



Proceedings of the 16 b Annual Conference of the International Academy for Information Management 



276 



TUTORIAL FOR A FRAMEWORK TO 
INTRODUCE THE EXTENSIBLE MARKUP 
LANGUAGE (XML) INTO INFORMATION 
SYSTEMS COURSEWORK 



Meg Murray 

Kennesaw State University 

Stephanie Cupp 
Kennesaw State University 



ABSTRACT 

New methods are required to efficiently and effectively handle the storage , retrieval , presentation and exchange of 
the vast amount of information electronically available. The extensible Markup Language (XML) was introduced 
to extend the capabilities of Web technologies to include a standardized way to exchange data. Consequently , the 
implications of XML permeate not only technology issues but also inter-business communications and industry 
standards in regards to data exchange. While interest in XML is high , the newness of the technology means there are 
few resources available for introducing XML into the IS classroom. This paper presents an introduction to XML and 
a short tutorial focused on basic XML vocabulary and elementary principles related to the creation of XML 
documents and associated DTD and schema technologies. Finally , A scenario is presented that can be used as a 
classroom exercise to provide students with the opportunity to explore the considerations and challenges of deploying 
XML technologies. 



INTRODUCTION 

As computing technologies expand, so does the 
collection of data. New methods are required to 
efficiently and effectively handle the storage, retrieval, 
presentation and exchange of the vast amount of infor- 
mation electronically available. SGML (Standardized 
Markup Language) was introduced more then twenty 
years ago to provide a framework for device independent 
representation of text in electronic form. An outgrowth 
of this framework was HTML (HyperText Markup 
Language). HTML provides an easy to use and 
standardized way to present multimedia information over 
the World Wide Web. The same international standar- 
dization body, the World Wide Web Consortium, known 
as W3C, that oversees the development of SGML and 
HTML introduced XML (extensible Markup Language) 



in 1996. The objective of the WC3 was to extend the 
capabilities of current Web-based technologies. XML is 
poised to be the next Internet standard for computer-to- 
computer exchange of data (Herman, 2001). XML has 
captured the attention of the business marketplace and 
research firms are touting it as the “Next Big Thing.” 
The percentage of businesses planning on implementing 
some aspects of XML in 2001 is expected to rise from a 
current 20% to 78%. E-commerce transactions using 
XML are predicted to rise to over 40% by the end of 
2003 (Patrizio, 2001). While interest in XML is high, 
the newness of the technology means there are very few 
resource materials available for introducing XML into 
the IS classroom. XML is multi-faceted; it is not another 
programming language. The implications of XML 
permeate not only technology issues but also inter- 
business communications and industry standards in 
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regards to data exchange. This tutorial will provide a 
framework for introducing XML into an information 
systems class and will include an exploration of XML 
and its corresponding components, technical 
implementation requirements, the development of 
Document Type Definitions (DTDs) or schemas for 
defining industry standard data definitions and the 
potential impact of XML on the field of information 
systems. 

Background Information on XML and 
Its Potential Importance 

XML is a meta-standard for defining other markup 
languages. Herman (2001) summarized what this means, 
“By itself XML does not provide any specific data 
standards. Instead, XML provides a standardized 
language for creating such standards. That’s why XML 
is called ‘extensible;’ it will be used by others to develop 
industry-or function-specific data definitions” (p. 27). 
The premise behind XML is that is separates data from 
presentation. With XML, systems can exchange struc- 
tured data, interpret that data, and display the data in any 
number of different ways. An XML stylesheet tells a 
Web browser exactly how to display the data in an XML 
document and an XML Document Type Definition or 
schema stores the definitions of tags related to industry 
specific data or fields of knowledge. Document Type 
Definitions (DTDs) or schemas, often referred to as 
dictionaries or vocabularies, serve as a uniform source of 
data definitions to promote the exchange of information 
between organizations within the same industry. 

The uses for XML are many. XML promises to be a 
neutral method for exchanging data between two 
systems or applications (Patrizio, 2001). The first major 
area of interest in XML resides within the business 
community. E-business is rapidly evolving due to the 
advances in technology that are continually underway; 
XML is intended to be a primary facilitator of the 
exchange of data between businesses involved in 
commerce such as B2B exchanges, consolidated supply 
chain methods for suppliers and vendors, and ease of 
Web site development for B2C companies. The current 
heterogeneous technology landscape characterized by 
multiple platforms employing various operating systems, 
database systems and software developed in multiple 
languages makes it difficult to facilitate the exchange of 
data between such systems. XML provides a common 
solution replacing the need for developing customized 
interfaces. The use of XML for databases is another 
large initiative. Databases are used in business. 



academics, search engine operations, and in virtually 
every other area of technology. XML can make the 
updating of data within databases considerably simpler 
by virtue of its design. XML makes use of object- 
oriented programming principles that allow for data 
templates to be created replacing the need to specify 
individual data items. XML also will be the basis for 
enterprise application integration (EAI) middleware 
(Herman, 2001), which will tie applications together 
across heterogeneous platforms including linking legacy 
systems to newer Internet based systems. Finally, XML 
will also “be key to handling the proliferation of new 
kinds of devices. Once data is encoded in XML, it can be 
combined with multiple stylesheets for formatting in 
different kinds of devices — from a PC to a PDA or 
Webphone. More radical transformations of the data also 
will be possible, such as from a text to a speech 
interface” (Herman, 2001). 

Challenges of XML 

XML is not without limitations and pitfalls. Those with 
technical backgrounds can draw on previously acquired 
skills to learn XML, but the learning curve nonetheless 
is steep. The technologies associated with XML are also 
far from complete. One major limitation is the lack of 
support from browser technologies. Both Microsoft and 
Netscape’s current browsers do not readily support XML 
at the level needed for easy application development or 
viewing. While XML assists in interoperability of data 
and applications, the development of different standards 
and DTDs and schemas by all types of industries and 
organizations is rapidly leading to confusion and a 
general lack of standards agreement. Kiely (1999) 
contends that XML’s flexibility can also be a downfall 
as “...it’s so flexible that two uncoordinated efforts to 
design schemas using XML produce incompatible 
results.” As often happens with the introduction of a 
new technology, misperceptions and hype about its 
capabilities abound. Clay Shirky (2000), summed up 
this up best when he described XML as a ‘Magic 
Problem Solver.’ According to Mr. Shirky, a “...Magic 
Problem Solver is technology that non-technologists 
believe can dissolve stubborn problems on contact” 
(Shirky, 2000). 

A primary benefit, and yet a major challenge for XML, 
will be related to interoperability. XML will facilitate the 
exchange of data but it will not happen as quickly as 
predicted. Shirky (2000) describes the “three sad truths” 
associated with XML. First, XML cannot design itself. 
People will still have to figure out how best to represent 
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their data, and then design tags that mark it up in the best 
way. Second, XML is not without pain. Implementing 
XML is time-consuming, costly, and often extremely 
complicated, especially when outdated legacy systems 
are involved. Finally, the politics involved in creating 
new standards and designing interoperable systems has 
nothing to do with the technology — it’s all about money 
and power. Everyone wants to control the latest, greatest 
thing, and when too many forces act on something in a 
plethora of different directions, it goes nowhere (p. 75). 

Even given these shortcomings, XML is positioned to 
thrive in the electronic marketplace as it facilitates 
intercommunications between businesses. Further, XML 
has the support of the major technology companies. One 
of the main premises behind Microsoft’s new .Net 
strategy is the belief that XML marks a turning point in 
the evolution of the Internet and computing architectures 
(Herman, 2001). While most expect that it will take a 
few years for XML to transcend the technology 
landscape, it is moving ahead at a rapid pace and will 
continue to mature into a key technology in the field of 
information systems. 

As XML is a new technology, few course materials are 
available to guide faculty in the introduction of this 
pervasive technology into existing IS courses. XML 
skill books that train students on the mechanics of XML 
coding are appearing on the market in mass. While the 
importance of acquiring an XML coding skill set should 
not be diminished, developing an understanding of the 
implications of XML as a technology that will transcend 
the development, implementation and use of information 
systems and transform business processes is paramount 
for the IS student. Facilitating the exchange of data and 
developing applications that incorporate dynamic 
presentation capabilities can be explained on a technical 
level in a straightforward manner. Understanding how 
these technicalities might be adopted and used is more 
complex. It requires an assessment of technologies and 
business processes, their interactions with each other and 
with the people who define them. XML further 
compounds the complexities of the people-process- 
technology triad by extending its reach beyond the 
internal organization to external entities who may even 
be competitors within the same industry.lt will be 
imperative for students in the field of information 
systems to develop a knowledge base related to XML, its 
capabilities, limitations, implementation and use. 



XML TUTORIAL 

The rest of this paper will be a short tutorial of the 
elementary principles for creating XML documents and 
corresponding resource files for DTDs and schemas. 
The tutorial will begin with an introduction to the basic 
vocabulary of XML and will end with a short scenario 
employing XML to exchange data between two 
computer systems. This tutorial can be used to present 
XML concepts in information systems or e-business 
courses that cover new technologies or issues related to 
systems integration and the sharing of data between 
software applications, or exchanging data between 
disparate database systems. 

XML Structure and Vocabulary 

In order to assess the potential impact of XML, it is 
necessary to understand the basic principles of XML. 
XML is multi-faceted and incorporates an expansive 
array of resources. XML is used to create ‘self- 
describing documents’ or a document that includes 
information describing what the document contains. As 
XML is a markup language, the ‘describing information’ 
is placed within tags. The advantages to this are 
numerous. First, XML is not limited to a fixed set of 
element types, or tags. For example, if XML is to be 
used to tag a document that describes a recipe, tags may 
be chosen to represent each ingredient as an item , or as 
an ingredient , or as groceries - the decision is made 
based on the project and preference of the developer. 

XML is more flexible than other commonly used markup 
languages such as HTML, the language used for Web 
pages. Consequently, HTML files look similar to XML 
files. However, the focus of HTML documents is to 
define how information is presented while the tags 
defined in XML documents describe the data and rely 
upon other technologies to handle the presentation of 
that data. This is advantageous when information is 
intended to be displayed in more than one manner. Once 
a tag has been used to define data within an XML 
document, the data can be used in a variety of ways. For 
instance, selected data from an XML document can be 
combined with data from another application or database 
and the combined data can then be presented back to the 
user. Search engines also provide better results on XML 
documents since the tags are descriptive of the actual 
data. 
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Further, XML documents are simply plain text files. 
This has many advantages. XML documents are easily 
readable both by people, programming languages and 
other applications. In addition, XML files are easily 
transferred across a network and can be passed using 
HTTP, the transport protocol of the Internet. 

XML technologies are structured around a collection of 
documents (files) and other associated resources. At the 
core, is the XML document. An XML document 
contains tags, elements and corresponding data or 
content. Associated with this XML document will be 
other files such as DTDs or schemas that specify what 
elements a document can contain or extensible 
Stylesheet Language (XSL) documents that describe 
how the data is to be displayed. These types of files do 
not contain data per se, but contain definitions or 
descriptions of how the data in the XML document 
should be handled.Other associated resources include 
Application Programming Interfaces (APIs) that allow 
programmers to make calls or interface with the XML 
document as it is being processed. 

The foremost rule of XML is that it must be well- 
formed.A well-formed document is one that is properly 
formatted and follows the basic rules of XML 
programming. One of the primary reasons that the 
language is extensible is that its rules are rigid. In order 
to use XML programs in many different ways, all of the 
programs must follow the same rules. Other markup 
languages do not require such standards, and thus tend to 
be restricted to one initial purpose or project. For 
example, most HTML documents tend to merge 
describing the data with describing the layout or format 
of the data. If these two ideas are interwoven, then the 
data can only be used for that purpose - it cannot be 
presented in another layout without editing the code. 
While the strict requirements of XML are sometimes 
seen as a disadvantage, well-formedness supports the 
core XML advantage of extensibility. 

Several initiatives are underway to develop standard 
specifications or vocabularies for applying XML to a 
variety of technologies. These efforts include 
incorporating XML with relational databases, using 
XML to define large documents and using XML to 
facilitate the exchange and presentation of discipline 
specific data. For instance, XML applications include 
the Chemical Markup Language for managing chemical 
data, the Mathematical Markup Language for 
Mathematical equations, MusicML for sheet music, 



Voice XML for the spoken work and ebXML which is 
a set of specifications designed to facilitate the exchange 
of business transactions over the Internet. 

While the resources associated with XML are many, a 
word of caution should be noted. XML is a young 
technology and many facets are still evolving and have 
not yet reach a standardized form. This presents many 
challenges to the XML developer. One goal of XML is 
to use the XML document as the foundation for data 
exchange and data presentation. XML as a medium for 
the exchange of data is operational. DTDs specifications 
were approved with the original XML specification. 
However, schemas, which are quickly becoming the 
preferred choice for data definitions, only reached 
approval in May of 2001. Data presentation is much 
more challenging. Current browsers do not fully support 
XML and it will be a while before these technologies 
converge. XSL, the style language for rendering 
presentation of XML data, is a proposed 
recommendation meaning that it is still under 
development and review. Until XSL matures to an 
accepted recommendation by the W3C, it will not be 
widely implemented. 

XML standards and technologies encompass their own 
vocabulary of terms and acronyms. Just as with learning 
any language, learning XML requires developing a basic 
understanding of its structure and familiarity with its 
primary lexicon. Following is a list of common XML 
terms and their associated meanings. 

Basic XML Vocabulary and Terms 

XML document - The XML document is the primary 
component of XML. The XML document contains 
elements, data or content and processing 
instructions. 

XML parser - An XML parser reads an XML 
document and verifies that its contents are well- 
formed. The parser will identify any error found but 
will not attempt to fix them. Browsers such as 
Microsoft’s Internet Explorer include an XML 
parser (but it does not check for validity.) In 
addition, several XML parsers have been written. 
Parsers are included with commercial XML 
development environments, included with books on 
XML or can be found on the Internet. Many are free 
such as Apache XML Project’s Xerces-J available at 
http://xml.apache.org/xerces-j/. 
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Document Type Definition (DTD) - A Document Type 
Definition specifies a set of rules that defines or 
constrains the contents of an XML document. 
Basically, the DTD is a coded list that identifies 
what tags in the XML document describe data and 
what constraints are put on that data. DTDs provide 
a common data definition and can be shared among 
many XML documents regardless of their location. 

Schema - The schema performs the same function as 
the DTD and is expected to be a replacement for the 
DTD. One primary difference between the two is 
that schemas are written in XML where DTDs have 
their own syntax. Another advantage is that the 
schema supports datatypes unlike the DTD that only 
supports strings. This requires a receiving appli- 
cation to convert the data to a different data type, 
such as a number, if necessary. The power of 
schemas is that they also can be shared by many 
different XML documents. Several initiatives are 
underway to develop industry specific schemas to 
facilitate the exchange of data between different 
organizations. 

Well-formedness - XML is a highly structured 
language and XML parsers make no attempts to fix 
any errors found within an XML document. 
Consequently, for XML documents to be usable, 
they must be well-formed. Well-formedness means 
that the XML documents adheres to the rules of 
XML. XML rules are defined in the XML 
Specification (World Wide Web Consortium, 2000). 

Valid XML document - DTDs and schemas contain 
data specifications. XML documents are compared 
against their corresponding DTD or schema in a 
process known as validation. A document must 
meet specific criteria in order to be valid. It must be 
well-formed, it must have a corresponding DTD or 
schema and it must meet all the constraints listed in 
the DTD or schema. Not all XML documents have 
a corresponding DTD or schema so will not need to 
be checked for validity and not all XML parsers 
check for validity. 

extensible Stylesheet Language (XSL) and XSL 
Transformation (XSLT) - XML separates data ( 
from its presentation. The XML technology used to 
render or present data contained in an XML 
document is the extensible Stylesheet Language 
(XSL). XSL contains a set of rules that are used to 



build templates that specify how XML document 
data are to be formatted for display. XSL is still 
under development. Consequently, no browser 
currently in use fully supports XSL. The current 
approach is to use an XSLT processor to transform 
XSL stylesheets into HTML output. 

Cascading StyleSheets (CSS) - Cascading StyleSheets 
provide another method for applying style to display 
data contained in XML documents. XSL provides 
more functionality than CSS but is harder to learn. 
CSS specifications were originally developed to 
work with HTML files but because of backward 
compatibility, CSS has not been widely imple- 
mented. As with XSL, not all browsers support 
Cascading StyleSheets. 

Xlink and XPointer - Xlink and Xpointer are evolving 
XML technologies that extend the use of linking 
documents or parts of documents together. Xlinks 
point to documents and Xpointers to individual parts 
of a document. The most common types of links 
used today are HTML link tags which link one 
HTML document to another. Currently these links 
are uni-directional pointing to a particular document 
at a particular location. Xlink supports multi- 
directional links and Xpointer allows links to 
individual parts of a document. 

XML APIs - XML documents themselves do not do 
anything. In order for them to be truly useful, data 
contained in these documents must be accessible by 
other applications. The software capable of reading 
XML documents and providing access to its content 
are XML Application Programming Interfaces 
(APIs). Developers can write their own APIs or use 
industry standard APIs. The two most common are 
the Document Object Model (DOM) and the Simple 
API for XML (SAX). 

Document Object Model (DOM) - The DOM provides 
a means for standardized access to XML documents. 
The DOM itself is not a program but a specification 
that implements a set of APIs that define how DOM 
compliant software can interface to XML 
documents. The DOM resides between the XML 
parser and the application requesting information 
from the XML document. The parser reads data 
from the XML document and feeds it into the DOM 
which is then accessed by the application. This 
method works by reading the entire document into 
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memory and constructing a tree of its contents. The 
major disadvantage to the DOM is its extensive use 
of memory. 

Simple API for XML (SAX) - SAX is an alternative to 
the DOM, but not necessarily a replacement. Like 
the DOM, SAX provides an interface between the 
XML parser and an application. SAX is event 
driven. Rather than storing the XML document in 
memory, SAX streams the document from start to 
finish looking for events that require action. SAX 
does not require the memory resources DOM does. 
However, for complex implementations, SAX may 
require several parses of the XML document. 

CREATING AN XML DOCUMENT 

XML is a markup language. A markup language is a set 
of rules and XML is a highly structured markup 
language meaning it requires its users to follow the rules 
explicitly. The basic components of a markup language 
are tags and elements. Tags are labels delimited by angle 
brackets while elements refer to tags plus their content. 



The basic rule of XML is that all elements must be 
surrounded by a beginning tag and an ending tag. 
Beginning tags are identified with the ‘<‘ and *>’ 
symbols while ending tags are identified with *</’ and 
‘>\ 

Beginning tag: <TITLE> 

Ending tag: </TITLE> 

Element: <TITLE> XML Tutorial </TITLE> 

An XML document is plain text and may be created in 
any text editor including Notepad which comes with 
Windows operating systems. XML documents are 
identified by a three-character filename extension of 
.xml. 

The basic building blocks of XML are elements and 
attributes. A complex XML document contains other 
components as well but this tutorial will be limited to the 
elementary principles of creating a simple XML 
document. Figure 1 provides an example of a simple 
XML document followed by detailed explanations of the 
annotations. 



FIGURE 1 

SAMPLE OF A SIMPLE XML DOCUMENT 



Prolog 



H <; ? xml Ys rsiQn^l.Q" CDCflding=''UTF-fiIfe- 



Comm en t j <{_s amp i e 0 f a simple XML document of a listing of Courses - 

Rrmt Element r 



Element w/Contcnt| ^ 
Empty Element Q 



1 <CwrscLis&_ 



Nested 



Element 



<Titte>lnformaaon Systems Courses > Fall Semester</Title> 






<Courso 

<Prefix>ClS</Pref]X> 

<Nurabcr>230Ck/Number> 

<Section>0 1 </Section> 

<CompNum> 1 0825c/CompNum> 

•cNamolntro to Computing</Name> 
cLocation Buildings”Pr Room="l 18"/> 
cSchedule Day=*MW" Time="08:00 * 01:45 PM7> 

</Cqutsc;> 



cCourse> 

<Prefix>ClSc/Prefix> 

<Numbcr>310O</Number> 

<Secdon>0 1 </Section> 

<CompNum> 1 0928</CompNum> 
<Name>Information Systems Management</Name> 

Elements with 
Attributes 



— — cLocation Building="KH" Room="l 1027> 

cSchedule Dav^S" Time="08:Q0 - 01:45 PM"/> 

</Course> 



cCourso 



<Prefix>C]Sc/Prefix> 

<N umber>3601 </Number> 

<Scctioa>0 1 </Section> 
cCompNuno 1 0936c/CompNum> 
cNamoSystems Analysis and Design</Naine> 
cLocation Building="KrT' Rotnrr= ,T l 1047> 
cSchedule Day= ,, TTH’ , Time=" 10:00 -11:15 PM H /> 
</Cour se> 

Clg?ingTP&fpr |</CourseList> 

Root Element 1 
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Description of the XML Document Components 

Prolog - The prolog contains the XML Declaration 
which is always the first line of any XML file. 
There should be nothing, including white space, 
before the declaration. The XML Declaration tells 
the processor which version of XML to use. A 
simple declaration may be: <?xml version = “ 1 .0”>. 
A complex declaration may include more infor- 
mation, such as: <?xml version = “1.0” encoding = 
“UTF-8” standalone = “yes”?>.”1.0” describes the 
version of XML being utilized (currently there is 
only one version of XML); “UTF-8” describes the 
language encoding such as English ASCII; and 
“yes” or “no” indicates whether or not the document 
relies on markup declarations defined external to the 
document. 

Comment - Comments may be included in XML files. 
They are ignored by the parser. Comments are 
included between the tags and 

Root Element -The most important component of the 
XML document is the element. All XML documents 
must have at least one element. The first element is 
known as the root element (also may be referred to 
as the document element) under which all other 
elements are nested. In Figure 1, the root element is 
‘CourseList.’ 

Element with Content - The most common format for 
elements is: <start-tag> content </end-tag>. This 
format is used for the majority of elements in Figure 
1. 

Empty Element - Elements with no content are known 
as empty elements. Empty elements are used when 
only the presence of the element is needed or the 
element contains only attributes. 

Nested Elements - Elements may contain other 
elements. This is referred to as nesting elements. In 
Figure 1, the Course element contains several other 
elements such as Prefix, Number, etc. In fact, the 
basic structure of XML documents is based on 
nesting all elements within the root element. The 
improper nesting of elements is one of the most 
common mistakes made when developing XML 



documents. If an element starts within another ele-ment, 
it must also end within that element. Table 1 shows the 
difference between incorrect and correct placement of 
end tags. 



TABLE 1 

EXAMPLE OF INCORRECTLY AND 
CORRECTLY NESTED ELEMENTS 



Correctly 
Nested Elements 


Incorrectly 
Nested Elements 


<Course> 

<Prefix>CIS 

</Prefix> 

</Course> 


<Course> 

<Prefix>CIS 

</Course> 

</Prefix> 




*The end tag for the Prefix 
element must come before the 
end tag for the Course 
element 



Elements with Attributes - Attributes are included in 
the beginning tag of an element and follow the 
format attribute = value. The value must always be 
enclosed in single or double quotes. An element 
may contain several attributes such as in Figure 1 
where the element Location contains two attributes, 
Building and Room and the element Schedule 
contains the attributes Day and Time. The decision 
to use an attribute or element is a matter of 
preference. A general rule, however, is to use an 
element for content that needs to be extracted 
individually and attributes for content not 
independently relevant. For instance the attributes 
Building and Room are not independently relevant 
but are directly related to the course Location. 

Closing Tag for Root Element - Generally the last line 
of an XML document is the closing tag for the root 
element. 

Well-Formedness of the XML document - Upon 
becoming familiar with the language, but prior to 
constructing an XML document, the rules for a well- 
formed document should be understood. 

• The XML Declaration must appear at the beginning 
of the document. 
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• The Root or Document Element must contain all 
other elements. The only statements allowed to 
appear before the Root Element are the XML 
Declaration, comments or processing instructions. 

• All elements must have a start tag and end tag. 

• Empty elements must either end with the /> or have 
both the start and end tags. 

• Elements must be properly nested. 

• Attribute values must be enclosed in quotes (double 
quotation marks (“) are the most commonly used). 

• XML is case sensitive; NAME and name are not the 
same. 

• Markup characters (< ,&,>, cannot be used 
within XML content. Instead their corresponding 
entity reference must be employed. The predefined 
entities for the mark up characters are &lt, &amp, 
&gt, &quot and &apos. 



Steps for Constructing the DTD and Schema 

The DTD is a coded list that identifies which tags 
describe data in the XML document. A DTD is not 
required to create an XML document; however, there are 
a few advantages to choosing to use a DTD. The DTD 
creates rules for large sets of documents to follow, which 
is important for projects with multiple authors, it defines 
what markup and what sequence should be used, and it 
provides a common frame of reference for many users to 
share. A properly used DTD will validate the document. 
A DTD document is plain text and DTD files are 
identified by a three-character filename extension of 
.dtd. 

The Document Type Definition is described in six basic 
components: XML Declaration, Document Type, 

Element Type, Attribute List, Entity and Notation 
declarations. Figure 2 shows the DTD for the Course 
Listing XML document displayed in Figure 1. 



FIGURE 2 

SIMPLE DTD FOR COURSE LISTINGS 



[ < IELEMENT CotuscList (Title, Cours^.liPLbcBins, Coursc>-)> 


Content 


< (ELEMENT Tilje (0PCOATA)> 


EMPTY r 


<!EUEMENT CtRmeJiujttfcim EMPTY :> 


1 


ANY 


<1ELEMENT Cutme (Prefix, Number. Section, CamjiNum. Name, Locatiuo, Scbodnfce)> 


klELEMENT IVefU ANY> 






<!ELEMENT Number ANY> 

< IELEMENT Section ANY> 

< IELEMENT CompNutn ANY> 




[< IELEMENT Name (4f PCD AT A )> 


Content 


< IELEMENT Uxa ik*i (*PCDATA» 
<IATTLIST Location 

Room CD AT A ^REQUIRED 
Building CD AT A ^REQUIRED 


Element with 


< IELEMENT Schedule (*PCDATA)> 
<IATTUST Schedule 


Attribute Ust 
Declaration 


Day (M | T | W | TH | F | S | MW | TTH) ^REQUIRED 
Time CDATA ^'REQUIRED 

^ 



i 
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The Element Type declaration defines the element type. 

There are four types of elements. 

• Element Content only contains child elements whose 
order is defined in the declaration. 

• ANY content may contain any well-formed XML. 

• EMPTY content may not contain anything. 

• Mixed Content elements may contain other elements 
in any order and also may contain text interspersed 
between elements. A special case of Mixed Content is 
Text Only where elements are defined to only contain 
text denoted by #PCDATA. No Mixed Content 
elements are included in Figure 2. An example would 
be: <!ELEMENT Name (#PCDATA | ChildName)>. 

Symbols in the element type declarations are used to 

control the occurrences of the elements. Table 2 

describes the symbols and their meanings. 



TABLE 2 

SYMBOLS IN ELEMENT TYPE 
DECLARATIONS 



Symbol 


Meaning 


> 


Listed child elements must be ordered as 
shown 


i 


Either one element or the other must be 
used 




(non-symbol) Required element 


+ 


Child element is used at least once 


★ 


The element is used as many times as 
needed or none 


7 


The element is used once or not at all 



The Attribute List declaration defines the names, 
datatypes, and default values of each attribute. The list 
includes all of the attributes that may be used within a 
given element. The format for declaring an attribute is 
be described by using an example. 

<!ATTLIST Course Level Enumerated Under | Grad 

#IMPLIED> 

• Course: Element Name 

• Level: Attribute Name 

• Enumerated: Attribute type 

• Under | Grad: possible values delimited by the | 
symbol 

• #IMPLIED: Default value 



Possible attribute types available in DTDs are fairly 
limited. The mostly commonly used are CDATA which 
allows the attribute to take on any string (text) value and 
Enumerated that defines a list of all possible values for 
that attribute. Other attribute types include ENTITY and 
ENTITIES that allow the insertion of non-text content 
into an XML document, ID and IDREF which allow 
unique identifiers to be association with an element and 
NMTOKEN and NMTOKENS which further constrain 
string values to a predefined character set. 

Possible default values for an element can also be 
defined in the attribute list declaration. These include: 

• #REQUIRED - the value must be supplied in the 
XML document. 

• #IMPLIED - the value may or may not be supplied. 

• #FIXED - the value of the attribute is given in the 
attribute declaration. 

• value - allows a value for the attribute to be assigned 
in the attribute declaration. 

Two other DTD declarations are available but will not be 
covered in this tutorial. The Entity Declaration defines 
a set of information that can be called simply by using 
the entity name. Entities can be thought of as short-cut 
keys. For instance, if the full name of an institution is 
frequently referenced, an entity might be created — 
<!ENTITY ksu “Kennesaw State University”> - so that 
&ksu might be typed in place of the full name. Any 
output would display the full text. The Notation 
declaration allows XML documents to refer to external 
information rather than duplicating data in the program. 
<!NOTATION Name-List SYSTEM “Names.exe”> for 
instance, refers, the program to the external ID, or file, 
called Names.exe. 

Using a DTD 

DTDs are called from within an XML document. A 
DTD may be used internally or externally depending on 
the project and preference of the authors. If the DTD is 
created externally, then it is referenced in the prolog 
immediately after the XML Declaration. For example, 
the first two lines of the XML document might be: 

<?xml version=”l .0 M > 

<!DOCTYPE CourseList SYSTEM “courselist.dtd”> 
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The Document Type declaration tells the processor the 
location of the DTD, such as a URL or a file path. For 
example, <!DOCTYPE CourseList SYSTEM 
“courselist. dtd”> indicates that the DTD called courselist 
is located locally in the current directory of the system 
being used. The first instance of CourseList in the 
example is the root element for the document. The 
second instance of courselist is the DTD file name. 

However, if the DTD is created internally, then the entire 
DTD is located in the prolog of the XML Document 
immediately after the XML Declaration within the 
Document Type Declaration. 

When DTDs are applied to an XML document, the 
content of the document is compared against the 
definitions in the DTD. XML documents that meet all of 
the constraints of the DTD, are said to be valid. Validity 
requires the XML author to fully understand the 
implications of the limits that are placed on any elements 
or attributes. Selection clauses should be used to choose 
one value from a set of possible values or to choose one 
element from a set of possible elements. Consider 
offering more than one possible order for data to be 
input in order to accommodate different types of data. 
For example, if an address is to be input, consider the 
different formats that addresses take in other countries 
by allowing for multiple ordering: 

<!ELEMENT Locator ((City, State?, Code?) | (Code?, 
City, State?))> 

Finally, more than one DTD may be applied to an XML 
document and more than one XML document can use a 
DTD. 

Schema 

Schema is an alternative to the DTD; although both 
techniques may be used together. The primary dif- 
ference is that the schema is written in the XML format 
and includes the use of datatypes. The schema provides 
several built-in datatypes as well additional derived 
datatypes. Some of the more commonly used datatypes 
include string, Boolean and those related to numbers 
(double, decimal, float) and time. Figure 3 depicts the 
schema for the XML document listed in Figure 1 and the 
DTD in Figure 2. 

Root Element - The XML format requires that a root 
element be established. In the XML Schema, the 
root element is ‘schema’ with an attribute that 



identifies a namespace. The prefix xsd is mapped to 
the namespace and used throughout the schema 
document. 

Namespaces - One advantage to XML technologies is 
the ability to share XML applications. One problem, 
however, is that the same tag name may be used in 
both applications but applied differently. 
Namespaces provide a way to associate elements 
and attributes to a specific XML application by 
mapping them to a particular URL. The URL does 
not have to point to a DTD or schema document. 
The URL is simply applied as a prefix making 
element and attribute names unique. Multiple name- 
spaces may be used in an XML document. The 
namespace declaration appears in the prolog of an 
XML file. The W3C schema specification, 
http://www.w3.org/2000/10/XMLSchema, is used in 
almost every XML Schema. 

The second root element is the root element of the 
schema document and usually matches the root element 
in the XML document. The type is a new type declared 
later in the schema. The new type contains the elements 
Course, Location and Schedule. The type is declared to 
be sequenced indicating the elements are expected to 
appear in the order presented. 

complexType - A schema may contain simpleType and 
complexType elements. A simpleType element is 
an element that only contains text and does not have 
any attributes or child elements. A complexType 
element may contain elements and/or attributes. 

simpleType with enumeration - The schema enumera- 
tion is similar to the enumerated component of the 
DTD attribute. Enumeration limits the element 
value to a choice from a list of specified values. 
Within this schema, enumeration is assigned to a 
simpleType named ‘class_time’ which is affiliated 
with the attribute ‘Day.’ 

simpleType with pattern - Pattern allows a format to 
be assigned to a value. Several options are available 
for pattern matching. In this schema, the 
simpleType is constrained to the format of two 
numbers, a colon, two numbers, a colon and two 
more numbers. To match the time datatype, an 
acceptable value would be 09:30:00 indicating hour, 
minute and seconds respectively. The simpleType, 
time_format is associated with the Time attribute of 
the Schedule element. 
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FIGURE 3 

SIMPLE SCHEMA FOR COURSE LISTINGS 



Root Element 



<?xml version = "1.0"?> 



Root Element 



<xsd:schema xmlns:xsd- l http://www.w3.orE/2Q0Q/10/XMLSchema' > > 



Namespace 



<xsd:eiemeat name= CourseList type=' L'ourseType 7> 
<xsa:compiexi>pe name= cx)urse_itsi_Degins > 
</xsd:complexType> 

:sd:complexType name="Uoursclypc ’> 



Empty Element 



New 1 



Define* 



complexType 


<xsd; complexType name="Couise”> 

<xsd:sequence> 

<xsd:element namesTrefix” type="xsd:string”/> 
<xsd:element name^Number” type^”xsd:positiveIntcger'7> 
<xsd:element name=" Section" type=“xsd:string”/> 


with Elements 


<xsd: element name="CompNum” type="xsd:positiveInteger7> 
<xsd:element name="Name" type="xsd:sinng7> 
</xsd:sequence> 

</xsd:complexType> 




<xsd:complex type name= Location > 

<x$d:sequence> 

<xsd:attributename="Bui!ding" type=* , xsd:string" use="required7> 
<xsd:attribute name="Room" type^’xsdrstrmg" use="required’7> 


complexType 


</xsd :sequence> 


with Attributes 


</xsd:complexT ype> 

<x$d: complexType name ="Schedule"> 

<xsd attribute name=’’Day" type="class„time" use="required"/> 
<xsd attribute name="Timc" type="tiroe_format" use="required”/> 
</xsd:complexT ype> 



</comp!exType> 



simple Type 


<xsd:simpIeType name=>"ciass_lime"> 
<xsdrrestrictkm base="x$d: string ”> 
<x$d:enumeration value="M''/> 
<xsd:enumeration value="T ,, /> 
<xsd:enumeration value="W"/> 
<xsd:enumeration value="TH"/> 
<xsd:enumeration vahie=T7> 
<xsd: enumeration vaiuc="S7> 
<xsd:enumeration vaiue^MW"^ 
<xsd:enumcratk>n valuer *TTir/> 
</xsd:rcstriction> 

</xsd:si mpleTypo 


with List 


simpleType 


<xsd:slmpItJType name=i UiucJfUnUiir > 

<xsd: restriction base="xsd:Titne"> 

<xsd:pattem value= "[0~9] { 2 } (: [0-9] { 2 })(: [0~9 ) { 2 })"/> 
</xsd:restriction> 


with Pattern 




</xsd:$impleType> 



OostnS teRl </xsd:schema> 
for root * 



Displaying XML 

Web browsers that support XML will not understand the 
tags in any given XML document (unless the tags 
happen to duplicate an HTML tag), so the style of 
presentation must be defined for the document. If no 
style is applied to an XML program, the browser will 
only show the text of the document. 

Two style languages are available for defining the 
presentation of XML documents. They include the 
extensible Stylesheet Language (XSL) and Cascading 
StyleSheets (CSS). XSL will provide a wide array of 
features and functionality for rendering XML content. 
XSL is still under review by the W3C and has not 
matured to a standardized form that can be adopted by 
browsers. CSS is supported, as least partially, by the 



commonly used browsers. It is still quite challenging to 
build stylesheets to display XML content. However, it 
is worthwhile to provide an example. CSS will be 
demonstrated in this tutorial. 

Just as is the case with the DTD, the CSS may be created 
externally or internally to the XML document. An 
external CSS is applied by declaring the CSS in the 
XML document between the XML Declaration and the 
DOCTYPE (DTD) declaration. 

<?xml version- T .0"> 

<?xml-stylesheet t y p e = ” text/css” 
href=”C:\css_CourseList.txt”?> 

<! DOCTYPE CourseList SYSTEM “courselist.dtd”> 
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Figure 4 is an example of a Cascading StyleSheet for 
displaying the course information contained in the XML 
document presented in Figure 1. Figure 5 shows the 
output with the style applied. 

The style assigned to the information can easily be 
changed by applying different stylesheets or by 
combining multiple stylesheets. For example, if an 
organization defines a specific look and feel for its 
Website, but each department within that organization 
wants to design details for their own Web page, multiple 
stylesheets can be used to allow them to achieve this 
objective. A guiding principle is to design an overall 
stylesheet that provides basic color schemes, layouts, 
headers, etc. Then design additional stylesheets that 
cooperate with the overall design but allow for additional 
details to be customized. 

XML Example Scenario 

XML includes many more facets than a short tutorial can 
cover. However, once a basic overview has been 
presented, considerations and challenges of deploying an 
XML solution can be explored. As a classroom exercise, 
scenarios can be presented and students can be asked to 
outline how XML might be used to facilitate better 



execution of the business process. Perhaps, the most 
difficult part in developing an XML application is 
designing the data model that will be used. Student 
groups can be assigned to work together to identify the 
data model and build a simple DTD or schema related to 
the problem under consideration. Students can then be 
asked, either individually or in a group, to construct 
corresponding XML documents. Finally an analysis can 
be made on how that schema might be utilized. An 
example scenario is presented below. 

Course Transfer Scenario 

An activity that frequently occurs within universities is 
the evaluation of courses for the application of transfer 
credits. For instance, a student takes a course at 
University A and then wishes to transfer the credit 
received for that course to University B. A method 
commonly employed to facilitate this is for the student 
to obtain a paper copy of their transcript from University 
A and submit that paper transcript for review to 
University B. This manual process is time consuming 
and often results in a delay before course credit is 
applied. An automated process would make the course 
transfer procedure more efficient.XML provides a 
solution. Figure 6 depicts how XML might be utilized. 



FIGURE 4 

EXAMPLE CASCADING STYLESHEET 



<?xml-stylesheet type-’ text/css" href=" c:\css_CourseList. txt" ?> 

<CourseList> 

<st id=”css_CourseListt”> 

Name {display: block; text-align: left; background-colongold; color:black;font-size: 100%; font- 
style: italic; font-family: sans-serif} 

Prefix+Number+Section{ display: block;font-size: 120%; font-style: italic; font-family: sans-serif} 
CompNum {display: none} 

Title {display: block; text-align: center; background-color: black; color: gold; font-size: 140%; 
font-style: bold; font-family: serif} 

</st> 

</CourseList> 
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FIGURE 5 

XML DOCUMENT WITH CASCADING 
STYLESHEET APPLIED 



Information Systems Courses - Fall Semester 



CIS 2300 01 



Intro to Computing 



CIS 3100 01 



Information Systems Management 



CIS 3601 01 



Systems Analysis and Design 



FIGURE 6 

APPLYING AN XML DOCUMENT TO THE 
COURSE TRANSFER SCENARIO 




Upon a request from the student, University A extracts 
information related to the courses the student has taken 
from its database and writes them to an XML document 
employing appropriate tags for each data item. The 
XML document file is then sent over the Internet to 
University B. University B reads the XML document, 
extracts needed data and inserts it into their database. 
Personnel at University B now have access to the 
courses taken by the student at University A for 
evaluation of course transfer credit. 

However, for this process to be successful, there is a 
caveat. University A and University B have to agree on 
the tags used in the XML document. This requires the 
two institutions to develop a proprietary agreement in 
relation to the contents of the XML document that both 
institutions must follow explicitly. For instance, a name 
can be presented in several ways; First-name, Last_name 
and Middle_initial as three separate elements or as three 
attributes of one element. The XML document created 
by University A would not be usable by University B if 
University A chooses to use attributes when University 
B was expecting elements.The XML solution to this is to 
develop a DTD or schema that is shared by both 



institutions. (In fact, this schema can be shared with 
other institutions as well). The schema provides the data 
definition for what can be contained in the XML 
document. Figure 7 depicts the process when a schema 
is used. 



FIGURE 7 

APPLYING A SCHEMA TO THE 
COURSE TRANSFER SCENARIO 




University A has access to the common schema and now 
the data that is extracted from their internal database is 
written to an XML document that follows the definitions 
in the schema. Further, the XML document is validated 
against the schema to insure compliance. The validated 
document is sent across the Internet to University B. 
University B knows exactly what data has been sent 
because they also have access to the schema. One major 
advantage of using the schema is that this process is 
extensible to other institutions given access to the 
schema. 

Within the classroom, students can be assigned to 
pseudo organizations and charged with the task of 
developing a schema or DTD for this scenario. Each 
group’s final product will be slightly different. 
Ultimately, however, only one common data definition 
can be used. (This usually translates to one common 
schema although the data definition could span multiple 
schemas.) The pseudo organizations Will need to work 
together to reach an agreement on one common schema 
or DTD. Students can then be assigned the task of 
developing individual valid XML documents. This 
exercises allows students to explore many of the 
challenges faced by organizations working to apply 
XML technologies. 

CONCLUSION 

XML and its associated standards have the potential to 
impact the way systems are developed and the way 
organizations share information. While expertise in 
XML syntax will not be necessary for all IS 
professionals, understanding how the technology works 
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and how the data is defined within that technology will 
be paramount to making informed decisions on the 
appropriate implementation of XML. Implementation 
will not be revolutionary; it requires an evolutionary 
approach that includes introspection of internal data 
models and reconciliation with data models from 
organizations to which information will be exchanged. 
XML should not be viewed as just another Web 
technology; but as a foundational technology that will 
transform the IT infrastructure. 

REFERENCES 

Apache Software Foundation (2000). “Xerces Java 
Parser.” Retrieved September 12, 2001 from http:// 
xml.apache.org/xerces-j/. 

Harold, E. R. (2001). XML Bible Second Edition. New 
York: Hungry Minds. 

Herman, J. (2001, April). “The XML Internet Takes 
Off.” Business Communications Review , 31(4), 26- 
30. 



Kiely, D. (1999, August 23). “BizTalk Could Spur XML 
and E-Business.” [On-line]. Information Week On- 
Line. Retrieved March 2, 2001 from the World Wide 
Web: http://www.informationweek.com/749/talk. 

htm. 

Patrizio, A. (2001, March 26). “XML Passes from 
Development to Implementation.” Information 
Week, 830, 1 16-120. 

Shirky, Clay. (2000, September 26). “XML: No Magic 
Problem Solver.” Business 2.0 , 1,75. 

World Wide Web Consortium (2000, October 6). 
“Extensible Markup Language (XML) 1.0.” 
Retrieved from the W3C Web site September 12, 
2001 from http://www.w3.org/TR/REC-xml. 



272 Proceedings of the 16 th Annual Conference of the International Academy for Information Management 



2&Q 



USING IT TO TEACH 

PROJECT MANAGEMENT IN THE MBA CORE 
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Extended Abstract 



We report on the evolution of a course in Infonnation 
Technology (IT) Project Management from an elective 
to a required core course in the MBA program at the 
University of Tampa’s John H. Sykes College of 
Business. This course is offered in the Information and 
Technology Management (ITM) department. 

In its current from the course, called “Effective Project 
Management,” is a seven-week course linked to the 
capstone course on Strategic Analysis. Students are 
required to take both courses in the same semester, 
Effective Project Management for the first seven weeks 
of the semester, and Strategic Analysis for the entire 1 4- 
week semester. In the Strategic Analysis course, student 
teams are required to act as strategy consultants to client 
companies in the Tampa Bay area. In addition, teams 
are required to use project management concepts and MS 
Project 2000 to manage their consulting projects. 

In January 2000, we offered this courseforthefirsttime 
as Special Topics in ITM elective. Most students took 
the course to fulfill requirements for a Concentration in 
Information Systems Management (ISM). We met for 
4.5 hours a night, five times a week for two weeks. We 
used a book by Kathy Schwalbe entitled “Information 
Technology Project Management.” The book came 
bundled with a 120-day trial version ofMicrosoft Project 
98. 

In its original form this course was described as a study 
of IT project management usingthe systems approach to 
problem solving. We covered all nine project 
management knowledge areas — project integration. 



scope, time, cost, quality, human resources, 
communications, risk, and procurement 
management — and all five process groups — initiating, 
planning, executing, controlling, and closing. We used 
Microsoft Project 98 to illustrate all concepts. 

On the first day of class we divided students randomly 
into teams of three or four, and we made random 
assignments of IT projects to groups. Lecture and 
discussion were limited to two hours per meeting. 
Teams used the remaining time to work on their projects 
using MS Project 98. We reserved the last meeting of 
class for team presentations. Project write-ups were due 
the following week. 

Student response was overwhelmingly positive that the 
course was offered four additional times, August 2000, 
January 2001, May 2001 and August 2001. In the 
meantime, the MBA program was redesigned to, among 
other things, add more value to the capstone experience, 
where student teams apply projectmanagement concepts 
and use MS Project 2000 to manage their strategic 
analysis consulting projects. 

Because of this MBA redesign, thecourse is now called 
“Effective Project Management.” It is to be offered 
every each fall and spring semester in the first seven 
weeks. In addition, the course is slightly changed to 
reflectthe fact that it is required of all MB A students and 
that it is linked to the Strategic Analysis course. We 
continue, however, to use the same book, albeit thelatest 
edition, which is bundled with MS Project 2000. One of 
the main reasons for continuing to use this book, an IT 
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Project Management book, is the fact that it is designed 
to reflect concepts and techniques in the Project Manage- 
ment Body Of Knowledge Guide 2000 as put forth by 
the Project Management Institute. 

Beginning fall 2001, we offer this course in its new 
format for the first time. In the remainder of this paper. 



we will examine the link to the Strategic Analysis 
course, describe the newly redesigned course in greater 
detail, explain why we use IT projects to illustrate 
project management concepts, explain why we continue 
to use the Schwalbe book, and report on initial student 
feedback. 
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ABSTRACT 

E-Business oriented management education programmes are currently founded all over the globe and becoming 
competitors for existing and successful MBA programmes , The primary o bjective of an MBA program me is to obtain 
top quality management training in the field of business, in a conductive environment with all the necessary resources 
available, and easy access to faculty , It has the most up-to-date teaching methods and the knowledge transmitted is 
state of the art , 

With this paper we like to present the results of a global survey which was carried out to analyse the various efforts 
being undertaken by MBA programmes around the globe to adapt their curricula with the advent of the E-Business 
age, The results of our survey — conducted for the second time after a two year period — will show a significant trend 
towards integration of E-Business relevant topics in existing majors versus the formation of new majors specializing 
in E-Business, 



INTRODUCTION 

MBA programmes around the globe provide graduate 
management training in the field of business/ 
management with the option to specialize in certain 
business areas. The training offered is most often 
intended to be very practical instead of highly 
theoretical. The curriculum is usually divided into two 
parts consisting of a core part, which covers the 
fundamental areas of business, (some of these courses 
may be waive d for peop le having p reviously studied the 
subjects in question) and an elective part, where students 
can take courses of special interest and perhaps 
specialize in a majo r field of business. Possible areas of 
interest offered range from finance, marketing and 



accountingto health care management, arts management, 
etc. [8], 

The need for changes to the content of MBA courses has 
grown extensively. The business world is changing 
rapidly every year. New business models are adding to 
already-existing market forces. Successful management 
in the e-business age is demanding new IT-based skills 
and IT-based knowledge. The first survey of this 
research project was conducted two years ago. Targeting 
more than 460 MBA programmes around the globe it 
showed quite clearly that the majority ofthe programmes 
had not integrated I S/IT applications and topics in their 
core curricula but planned to do it in the future [9] [10]. 
Concerning applicationskillsthemaj ority of programme 
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directors believed that these skills should not be taught 
as part of the programme at all. With regard to all 
Internet related topics the findings of the survey 
confirmed the hypothesis that there was a significant 
change in the importance of curriculum content. In 
expanding our research project we decided to carry out 
a follow-up survey covering the most interesting 
question of the integration of the new content in the 
MBA curriculum core. We tried to gain more insight into 
the MBA curricula changes to find whether these IS/IT 
related topics which were getting more important in the 
future were being integrated in existing core courses or 
were being taught in enrichment and elective courses 
(coveringproblems like multidisciplinary or cross course 
integration). 

MANAGEMENT SKILLS 
FOR THE E-BUSINESS AGE 

During the last years, significant efforts havebeen made 
in the IS field to put forward general recommendations 
for curriculum guidelines for Master Programmes in 
Information Systems. These efforts were being 
spearheaded by the joint ACM/AIS Curriculum 
Committee on IS Curriclula.This group suggests [6] that 
the output of any new programme should create people 
with the following characteristics. 

• Broad business and real-word perspective 

• Communication, interpersonal, and team skills 

• Analytical and critical thinking skills 

• Integrated IT and business foundations, a core of IS, 
and specific skills leading to a career. 

As many important business issues are nowadays at 
least to some extent IS issues it is quite clear that 
efforts are being required to adopt current curricula. 
Figure 1 shows Wigands, Picots and Reichwalds 
assumption of the potential for innovation, competitive 
markets and innovation strategies. 

Nevertheless, the vast majority of general management 
jobs still have a strong business profile and career track 
(marketing & sales, finance, human resource 
management, etc.). Today the number of new market 
challenges is also placing new essential demands on the 
manager such as mastery of newer information and 
communication technologies [i]. 



As a result MB A curricula are also having to adaptto the 
challenges and requirements of the e-business age. The 
modern MBA graduate should be a general manager, 
capable of refreshingacompany, communicating visions 
and goals. He has to be flexible and open to new ideas 
and business models [7]. With regard to skills and 
knowledge an MBA graduate should know about the 
relevant business potential of ERP systems and the 
Internet, know how to use executive information 
systems, have at least a strong knowledge of e-business 
issues and e-business based implications on different 
managerial areas. By collecting data from and combining 
the results ofdifferent research projects and surveys [5] 
[1] [4] [3] [2] [13] [11] [12] we had created a checklist 
of prerequisites of I S/IT -skil 1 s and IS/IT related 
managerial topics for the e-business age. 

Together with the data collected from our first global 
survey we derived different clusters ofpossible relevant 
E-Business related curriculum content. Based on thatwe 
searched for a matching fram ework of param eters which 
should help us in operationalizing our hypotheses. We 
used the framework of Werthner and Klein [14] to 
identify the relevant parameters influencing the building 
of an E-Business strategy as well as relevant external 
parameters for management decisions in the area of E- 
Business. 

RESEARCH QUESTION AND HYPOTHESES 

Given the knowledge that MBA programme managers 
plannedto adapt theircurricula andwantedto includeE- 
Business relevant content and monitoring the ongoing 
various economic changes and technological 
developments we therefore wanted to analyze whether 
these E-Business related topics which were getting more 
important in the future were being integrated in existing 
core courses or were being taught in enrichment and 
elective courses (covering problems like 
multidisciplinary or cross course integration) or were 
still being neglected. 

To analyze the situation we formulated the research 
question stated above taking into account some of the 
necessities and prerequisites to adapt MBA curricula in 
order to successfully react to market needs in the E- 
Business age. 
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FIGURE 1 

CHANGES IN COMPETITIVE ENVIRONMENT AND CORPORATE STRUCTURES [15] 




FIGURE 2 

E-BUSINESS RELATED PARAMETERS INFLUENCING MANAGEMENT DECISION MAKING 
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We then derived a main hypothesis to be tested in a 
survey covering the same MBA programmes of our first 
global survey. 

MBA programmes have identified and reacted to the 
new managerial challenges and skills required for E- 
Business. 

To find answers to our research question we 
operationalized our main hypothes is by formulating six 
hypotheses to be tested during our study: 

If MBA programmes have identified and reacted to the 
new managerial challenges and skills required for the E- 
B us in ess age then: 

• the majority of all MBA programmes have or will 
establish special E-B usiness majors within their 
curricula (Hypothesis 1) 

• the majority of all MBA programmes will increase 
the coverage of E-Business related technology 
infrastructure in their curricula in the future 
(Hypothesis 2) 

• the majority of all MBA programmes will increase 
the coverage of E-Business related organizational 
applications in their curricula in the future 
(Hypothesis 3) 

• the majority of all MBA programmes will increase 
the coverage of E-Business related policy issues in 
their curricula in the future (Hypothesis 4) 

• the majority of allMBA programmes will expand the 
use and teaching of IS/lT-skills in their curricula in 
the future. (Hypothesis 5) 

• the majority of all MBA programmes will increase 
the overall coverage of E-Business based 
applications and E-Business based topics as 
electives in their curricula in the future (Hypothesis 
6 ) 

RESEARCH METHOD- 
DESIGN OF THE STUDY 

The research team decided that a survey with a detailed 
questionnaire would be the most appropriate method of 



collecting the required data to find answers to our 
research questions and to test our hypotheses. 

MBA programmes all around the globe were defined as 
the unit of analysis of this part of the survey. As far as 
possible programme directors were addressed directly. 
Prior to sending out the questionnaire a pre-test was 
carried out with selected European MBA programme 
directors which should resemb le the actual respondents 
for the survey. After conducting the pre-test we had 
identifiedquestions whichhad been difficult to interpret; 
we clarified terms and definitions; we eliminated 
unnecessary questions and had identified the time 
required to go through all the questions. 

The final version of the questionnaire was sent out to 
466 MBA programmes around the globe. After the 
deadline 121 completed questionnaires had been 
returned, giving an average response rate of 25,96%. 
Figure 3 is showing sample response rates for different 
global regions of our survey (During our 1998 survey 
102 questionnaires had been returned with a response 
rate of 2 1,89%.). 

FINDINGS OF THE STUDY 

In Figure 4 we present some sample findings of our 
study concerning the hypotheses developed according to 
ourresearch question. Thetesting of the hypotheses gave 
a goodpicture concerningthe preparations and adoptions 
of MBA curricula for the electronic business age. For 
each hypothesis there was a number of operationalized 
questions being included in the questionnaire. The 
following section is showing aggregated and clusterd 
results. 

Hypothesis 1 

The majority of all MBA programmes have or will 
establish special E-Business majors within their 
curricula (Hypothesis 1). 

As Figure 4 is showing the majority of all MBA 
programme directors 72.1% have not established or 
are not planning to establish special E-Business majors 
in the future. 
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SURVEY RESPONSE RATE 





Africa 


Australia & 
Asia 


Europe 


North 

America 


South 

America 


Total 


1998 Sent out 
questionnaires 


11 


113 


115 


136 


91 


466 


2000 Sent out 
questionnaires 


11 


113 


115 


136 


91 


466 


1998 Returned 
questionnaires 


3 


16 


37 


32 


14 


102 


2000 Returned 
questionnaires 


5 


27 


35 


41 


13 


121 


1998 

Response rate 


27,27% 


14,16% 


31,18% 


23,53% 


15,38% 


21,89% 


2000 

Response rate 


45,45% 


23,89% 


30,43% 


30,14% 


14,28% 


25,96% 



FIGURE 4 FIGURE 5 

E-BUSINESS MAJORS E-BUSINESS RELATED 

TECHNOLOGY INFRASTRUCTURE 

Establishment of E-Business Majors 



Yes 


127 


27,26% 


No 


336 


72,10% 


N. A. 


3 


0,64% 


Total 


486 


100,00% 



Hypothesis 2 

The majority of all MBA programmes will increase the 
coverage of E-Business relatedtechnology infrastructure 
in their curricula in the future (Hypothesis 2). 

Hypothesis 3 



Increase the coverage of E-Business related 
technology infrastructure 



Yes 


289 


62,02% 


No 


170 


36,48% 


N.A. 


7 


1,50% 


Total 


466 


100,00% 



Hypothesis 4 

The majority of all MBA programmes will increase the 
coverage of E-Business related policy issues in their 
curricula in the future (Hypothesis 4). 



The majority of all MBA programmes will increase the 
coverage of E-Business related organizational 
applications in their curricula in the fiiture (Hypothesis 
3 ). 
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FIGURE 6 

E-BUSINESS RELATED 
ORGANIZATIONAL APPLICATIONS 



Increase the coverage of E-Business related 
technology infrastructure 



Yes 


391 


83,91% 


No 


71 


15,23% 


N. A. 


4 


0,86% 


Total 


466 


100,00% 



FIGURE 7 

E-BUSINESS RELATED POLICY ISSUES 



Increase the coverage of E-Business related 
technology infrastructure 



Yes 


372 


79,83% 


No 


92 


19,74% 


N.A. 


2 


0,43% 


Total 


466 


100,00% 



Concerning special E-Business related topics covering 
the areas oftechnology infrastructure (www technology, 
hardware, software, applications, payment systems, 
authentication and security systems, etc.), organizational 
applications (business models, operational and strategic 
perspectives, pricing, marketing, logistics, etc.), and 
policy issues (legal and policy issues, privacy, content 
selection and rating, intellectual property rights, tax 
implications, etc.) figures 5 to 7 show a significant 
increase in the coverage in all three areas. 

Hypothesis 5 

The majority of all MBA programmes will expand the 
use and teaching ofIS/IT-skills in their curricula in the 
future (Hypothesis 5). 



These numbers Support a very homogeneous picture 
which we already identified and painted after our last 
survey. The majority of MBA programmes do not want 
IS/IT skills to be taught as part of the programme at all. 
Therefore it should be no wonder that the vast majority 
of programme directors are not planning to expand the 
use and teaching of IS/IT skills in their curricula. 

FIGURE 8 

EXPANDING THE USE AND 
TEACHING OF IS/IT-SKILLS 



Expanding the use and teaching of IT/IS-skills 



Yes 


109 


23,39% 


No 


355 


78,18% 


N.A. 


2 


0,43% 


Total 


466 


100,00% 



Hypothesis 6 

The majority of all MBA programmes will increase the 
overall coverage of E-Business based applications and 
E-Business based topics as electives in their curricu la in 
the future (Hypothesis 6). 

FIGURE 9 

E-BUSINESS ELECTIVES 



Increase the overall coverage of E-Business 
based applications and topics as electives 



Yes 


419 


83,91% 


No 


43 


9,23% 


N.A. 


4 


0,86% 


Total 


466 


100,00% 



By asking about the roll of electives as possible solution 
for increasing the overall coverage of E-Business based 
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applications and topics the survey showed a significant 
positiv trend. Nearly 90% of all MBA programme 
directors are plannin g to increase the offer of E-Business 
related electives. 



CONCLUSIONS 

During the testing of our six hypotheses four of them 
could be confirmed as can be seen in Figure 10 giving 
us a more or less clear answer to our overall research 
question. This study shows that there is significant effort 
to change and adapt MBA curricula in E-Business 
related areas. The results of our survey show a 
significant trend towards integration of E-Business 
relevant topics in existing majors and an increase in the 
supply of E-Business related electives versus the 
formation of new majors specializing in E-Business. 
Concerning all Internet related topics the in-depth 
findings of the survey did not show that there was a 
significant change in the importance of curriculum 
content compared to the survey of 1998. 

FIGURE 10 

HYPOTHESES TESTING SUMMARY 



confirmed not confirmed 



Hypothesis 1 
Hypothesis 2 
Hypothesis 3 
Hypothesis 4 
Hypothesis 5 
Hypothesis 6 




Expanding our research project in the near future we will 
try to compare our findings with a planned survey 
covering the vast number of specialized E-Business and 
E-Commerce programmes which have been and 
currently are established all overthe globe. So far there 
are no clear boundaries for outlining the curriculum to 



make it clear what content is taught in which 
programmes and courses. 
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ABSTRACT 

Distance learning is quickly becoming an accepted and even necessary part of college and university programs. As 
more colleges and universities join the growing ranks of institutions offering distance learning, educators and 
administrators are struggling with the issue of how to assess student success in this new and largely untested 
environment. Many distance learning providers, and even some institutions, have developed short surveys that are 
designed to gauge whether a student is prepared to undertake distance learning. These tests, however, may not 
accurately assess a student's predisposition and learning style. This paper highlights and compares the use of 
different learning style inventories as a means to formally and empirically assess learning styles. Students in both 
distance learning and traditional classroom courses were given several of these inventories and their progress was 
tracked. Initial results indicate that some of these can be used as a successful predictor of student performance and 
may be useful for students and administrators in determining whether or not the student should undertake a distance 
learning course or program. The paper concludes with some suggestions and implications for educators on distance 
learning. 



INTRODUCTION 

Distance learning has become an established, accepted 
part of many college and university offerings and, in 
some cases, represents a major portion of the curricula. 
Given the technological advances and broad reach of the 
Internet, this is not surprising. Students and admini- 
strators alike are demanding such programs given the 
changing student demographics and societal needs. As 
the traditional college populations change, the need for 
programs which address convenient, flexible, and 
adaptable learning increases. Life-long learning and re- 
tooling also necessitate programs which can 
accommodate full-time workers and those with familial 
obligations. Need notwithstanding, distance learning 
providers have not spent much time investigating 
whether or not this new pedagogy is suitable for 
everyone and how to assess who might benefit from it. 

Many institutions offer distance-learning courses and/or 
programs but have little or no pre-assessment for 
students. They feel that the need for such courses and 
programs is justified, however they often do not provide 



a way for prospective students to assess their level of 
readiness for such programs. This is especially true for 
those students who have been away from the classroom 
environment for some time and are now returning to 
continue a course of study or re-tool in a new area. This 
paper begins with an exploration of some of the ways in 
which students are currently tested, highlights and 
compares several different learning style inventories, and 
presents initial findings from both on-campus and on- 
line classes with respect to student learning styles. It 
concludes with general suggestions for educators and 
administrators on how to assess student readiness for this 
new environment. Future research will gather data from 
a much larger population to assess which learning style 
assessment, if any, is most suitable for assessing student 
readiness for this new and largely unexplored pedagogy. 

EXISTING ASSESSMENTS 

A few of the commercial distance learning providers 
have developed their own self-assessment surveys or 
questionnaires to assist students in deciding whether or 
not to undertake a distance learning course or program. 
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Some are very short, others are more comprehensive and 
thorough. For example, one of the providers, 
eCollege™, uses a short 10-question quiz to assess 



potential student success (http://www.onlinecsu.ctstateu. 
edu/index.real?action-IsOnline). 



« Student/FncuKy Login 



- About OniineCSU 
» Course Catalog 
® Registration 



« Financial Information 
a Degree Information 
w Accreditation 
» Academic and 
Student Servicer, 



» Student FAQs 
o Facuity/Staff FAQs 



® Technical Information 
» Browser Test 
o Help Desk 
* Inquiry 
« Site Map 



UnM to Ail Four 
CSU Universities 



msm 

EftMSp 



CSU... Dew toping a 
Stoto of Minds 



Is Online Learning For Me? 

Take the Quiz... 

1. My need to take this course is: 

O high- I need it immediately for a degree, job, or other important reason. 

(* moderate- I could take it on campus later or substitute another course. 

O low- it is a personal interest that could be postponed. 

2. Having face-to-face interaction is: 

O not particularly important to me. 

<* somewhat important to me. 

C very important to me. 

3. I would classify myself as someone who: 

O often gets things done ahead of time. 

(?) needs reminding to get things done on time. 

O puts, things off until the last minute. 

4. Classroom discussion is: 

O rarely helpful to me. 

@ sometimes helpful to me. 

C almost always helpful to me. 

5. When an instructor hands out directions for an assignment, I prefer: 

O figuring out the instructions myself. 

(!) trying to follow the directions on my own, then asking for help as needed, 

O having the instructions explained to me. 

6. I need faculty to constantly remind me of due dates and assignments: 

O rarely. 

(?) sometimes, 

O often. 

7. Considering my professional and personal schedule, the amount of time I have 
to work on an online course is: 

O more than for a campus course. 

(? the same as for a class on campus. 

O less than for a class on campus. 

8. When I am asked to use email, computers, or other new technologies 
presented to me: 

O I look forward to learning new skills, 
fly I feel apprehensive, but try anyway. 

O I put it off or try to avoid it. 

9. As a reader, I would classify myself as: 

C good- I usually understand the text without help. 

(?) average- I sometimes need help to understand the text. 

O below average- I often need help to understand the text. 

10. If I have to go to campus to take exams or complete work: 

C I have difficulty getting to campus, even in the evenings and on weekends. 
p I may miss some lab assignments or exam deadlines if campus labs are not 
open evenings and weekends. 

C> I can go to campus anytime. 
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The results of the exam, given a “middle of the road” set of responses as indicated above, are: 




* Welcome 

* Student/Faculty Login 

* About OnllneCSU 

* Course Catalog 

* Registration 

* Financial Information 
» Degree Information 

* Accreditation 

* Academic and 
Student Services 

* Student FAQs 

* Faculty/Staff FAQs 

* Technical Information 

* Browser Test 

* Help Desk 

* Inquiry 

* Site Map 



Link to All Four 
CSU Universities 




CENTRAL 

coMKEcnorr 




PKtT V 



Results 



You scored: 20 





0 



\ Western 
iOl| \ Connecticut 
; State 
University 



20 points or higher- an online course is a real possibility for you. 

Between 11 and 20 points- an online course may work for you, but you may 
need to make a few adjustments in your schedule and study habits to succeed. 
Less than 10 points- an online course may not currently be the best alternative 
for you; talk to your counselor. 

No matter what you scored, remember that online learning is not easy. Your 
professor will demand at least the same quality of work as they would receive 
from you in a face-to-face classroom. A similar number of hours will need to be 
committed throughout the course of a semester for an online course as to a face- 
to-face course. Remember that your course may include deadlines and 
instructions on assignments, but there will not be anyone telling you to "turn in 
your assignment." Online learning is convenient- you do not have to commute to 
campus, and you can attend class at your convenience- either early in the 
morning, late at night, or anytime in-between. Just remember, no matter when 
you decide to study, your professor will hold you accountable- so study hard, and 
have a great term! 



eCollege 

(http://www.onlinecsu.ctstateu.edu/index.real?action=IsOnline) 



§1 

life 

@ 



ANALYSIS 

While the results may be helpful to a student, the point 
system illustrated above is vague and a variation on only 
one or two answers could result in a recommendation 
against distance learning. While the eCollege™ survey 
is easy to take, the accuracy of the results might be 
questionable in that students are not asked specific 
questions about their learning styles but rather more 
about the environment (e.g., their level of comfort with 
technology, ability to visit campus, and/or need for 
contact with the instructor). 

It is interesting to note that as of mid-2001, national 
institutions such as the University of Phoenix™, Jones 
International University™ and Western Governors 
University™ do not offer any type of pre-assessment. 
They encourage the student to enroll and work with a 



counselor and/or technical support personnel to bring 
them up to speed in their environment. This is a 
common approach, but one that may subject the student 
to unnecessary stress as they attempt to work out their 
difficulties and determine the suitability of a distributed 
learning environment. 

So how does a student determine if he/she will be 
successful in a distance-learning environment? They 
could enroll in one of these national or local institutions 
and “learn the ropes” and hope that they will be able to 
handle the rigors of learning without the benefit of an 
actual classroom setting. For older students and those 
who already have substantial work experience and/or a 
previous degree, this may not be a difficult adaptation. 
Students who will potentially require more assistance are 
those whose learning styles may not be amenable to a 
non-classroom setting. Local and regional institutions in 
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particular may target their own students or those in the 
local area rather than a national or international clientele. 
It is for these institutions that a more comprehensive 
assessment of learning style is beneficial (Sternberg, & 
Grigorenko, 1997). 

OVERVIEW OF LEARNING INVENTORIES 

There are numerous learning style assessments currently 
available. Some were designed for learning assessment 
in general while others have been adapted or modified to 
accommodate many of the newer learning styles and 
pedagogies. 

One of the oldest and most well known ways of 
assigning students to a given learning style is that of 
Kolb’s (1984) Learning Style Inventory (LSI). This 
assessment inventory consists of 36 words in 9 
groupings of 4 each. The student is asked to rank each 
of the sets of words on a 1 to 4 scale, with 1 equating to 
least like the person, 4 being the most like the person. 
The four columns of words correspond to four learning 
style scales: concrete experience (CE), reflective 

observation (RO), abstract conceptualization (AC), and 
active experimentation (AE). Kolb uses Jung’s (1977) 
typologies as the main foundation in the development of 
these learning styles. For example, the abstract 
conceptualization (AC) style “focuses on using logic, 
ideas, and concepts. It emphasizes thinking as opposed 
to feeling” while the concrete experimentation (CE) style 
“focuses on being involved in experiences and dealing 
with human situations in a personal way. It emphasizes 
feeling as opposed to thinking” (Kolb, 1984, 68-69). 

Kolb’s LSI has been criticized for its low reliability and 
validity measures (Freeman & Stumpf, 1978; Holman, 
Pavlice & Thorpe, 1997; Lamb & Certo, 1978; West, 
1982), yet it has received equal support as a way of 
illustrating the different approaches to learning (Abbey 
et al, 1985; Kruzich et al, 1986; Nulty & Barrett, 1996; 
Raschick et al, 1998). Asa result of the diverse opinions 
and findings, other methodologies are needed to provide 
a more stable platform with respect to validity and 
reliability of such an assessment. Other methodologies 
which will be included in this investigation include Tait 
and Entwistle’s ASSIST (1996), Solomon and Felder’s 
(1996) Index of Learning Styles (ILS), Honey and 
Mumford’s (1992) Learning Styles Questionnaire (LSQ) 
and the Academic Self-Efficacy Scale (Eachus, 1993). 

The ASSIST (Approaches and Study Skills Inventory for 
Students) instrument developed by Tait and Entwistle 



(1996) is a 38-item inventory which attempts to identify 
students with weak study strategies. It has four 
sub scales which measure four approaches of studying 
and academic aptitude. The scales are deep (intention to 
understand, relation of ideas, active learning), surface 
(intention to reproduce, unrelated memorizing, passive 
learning), strategic (study organization, time 
management, intention to excel), and apathetic (lack of 
direction and interest). Students respond to items 
relating to each of these approaches along a five-point 
likert scale from “agree” to “disagree”. A score for each 
of the approaches is determined by summing the scores 
from each of the items corresponding to each subscale. 

Solomon and Felder’s (1996) Index of Learning Styles 
(ILS), originally developed for engineering students, 
focuses on four bi-polar preference for learning scales. 
These include Active-Reflective, Sensing-Intuitive, 
Visual-Verbal, and Sequential-Global (Felder & 
Silverman, 1988). Active learners are those who learn 
by trying things and working with others. Reflective 
learners prefer to think things through and work alone. 
Sensing learners are oriented toward facts and 
procedures while Intuitive learners are more conceptual, 
innovative and focus on theories and meanings. Visual 
learners prefer visual representations of material such as 
pictures, diagrams and charts while verbal learners prefer 
written or spoken explanations. Sequential learners are 
linear and orderly in their thinking and learn in small 
incremental steps while Global learners are holistic 
thinkers who learn in large leaps. These bi-polar scales 
offer a good basis for comparison of learning types. 

The Learning Styles Questionnaire (LSQ) developed by 
Honey and Mumford (1992) identifies four types of 
learners, Activists (e.g. enjoy new experiences, make 
intuitive decisions, dislike structure), Theorists (e.g. 
focus on ideas, logic and systematic planning, mistrust 
intuition), Pragmatists (e.g. favor practical approaches, 
group work, debate, risk-taking), and Reflectors (e.g. 
observe and describe, try to predict outcomes, try to 
understand meaning). According to the authors, 
individuals tend to rely on one of these approaches when 
they are engaged in learning. 

Finally, the Academic Self-Efficacy Scale (Eachus, 
1993) is a 23-item scale which assesses the extent to 
which students believe they have the ability to exert 
control over their academic environment. By totaling 
the scores from the items, a self-efficacy score can be 
determined. This can be useful for students to determine 
the extent to which a distance-learning environment will 
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be suitable for them. By knowing their level of self- 
efficacy, they will be in a better position to make a 
decision as to whether or not to pursue a more traditional 
course of study. 

METHODOLOGY & PRELIMINARY RESULTS 

Each of the assessment inventories highlighted above 
were administered to students in an on-campus 
sophomore level computer class focusing on hardware, 
software and networking. The same inventories were 
administered to students in a distance-learning course 
conducted during the summer of 2001. The course 
covered the same content as the on-campus course, but 
was delivered entirely over the Internet. 

Preliminary analysis regarding each of the learning 
styles is presented below. A total of 40 students took 
part in this pilot study. 25 took the traditional on- 
campus class and 15 took the distance-learning class 
over the summer. The mean age was 19 for the on- 
campus class and 20 for the distance-learning class. 
There were 15 women and 10 men in the on-campus 
section and 1 1 men and 4 women in the on-line section. 
All were classified as full-time students although 80% 
reported working at least 20 hours per week while they 
took the courses. It must be noted here that this small 
sample size prevents the results that follow from being 
generalized to the general population. It is the author’s 
intention to undertake a full study given the promising 
results of this initial pilot study. 

Kolb categorizes four learning styles: Concrete 

Experience (CE), Reflective Observation (RO), Abstract 
Conceptualization (AC), and Active Experimentation 
(AE). Analysis of the sample indicates that 75% of the 
students responded as Abstract Conceptualizes (AC) 
and 85% classified themselves as Active Experimenters 
(AE). Given the subject of the course, computer 
hardware and software, this is not surprising. It is 
interesting to note that more women (13%) then men 
(7%) categorized themselves as Reflective Observers 
(RO). This might be indicative of male-female 
personality differences, but this cannot be verified given 
the sample. 

Analysis of Tait and Entwistle’s (1996) ASSIST 
(Approaches and Study Skills Inventory for Students) 
instrument indicates that men (12%) are more inclined to 
be surface learners (e.g. more passive and inclined to 
memorize) than women (8%). Women outnumber men 



almost two-to-one in terms of deep (active learning, idea 
relation) learning (28% to 15%). Few students indicated 
an apathetic learning style and most were highly focused 
on strategic learning. Incidentally, this focus was much 
more pronounced in the on-line class (82%) than in the 
on-campus class (68%), possibly suggesting a greater 
need for time management and organization. This may 
have been the result, however of the shortened (six- 
week) class time period. 

Solomon and Felder’s (1996) Index of Learning Styles 
(ILS) focuses on four bi-polar scales. Active Learners 
try things out and prefer working with others while 
Reflective learners prefer to work alone and think things 
through. Students in both classes were largely 
Reflective learners, however this may be the result of 
their age and lack of experience with group settings. 
Sensing learners are more fact-oriented and Intuitive 
learners are more conceptual. Not surprisingly, students 
were more Sensing (87%) than Intuitive (56%), probably 
a result of their chosen major, computer information 
systems. People with an aptitude toward computers tend 
to be sensing and thinking which is in line with the 
Myers-Briggs Type Indicator. Students were also more 
Visual (92%) then Verbal (8%). This may be largely the 
result of age. Generation Y has grown up in a much 
more visually-oriented world and results may differ 
greatly if the population were not so homogeneous. 
Also not surprising is the all the students identified as 
being Sequential learners. This again may be a direct 
result of age. 

The Learning Styles Questionnaire (LSQ) developed by 
Honey and Mumford (1992) identifies four types of 
learners: Activists, Theorists, Pragmatists, and 

Reflectors. Students identified largely as Theorists 
(72%) and to a lesser degree as Reflectors (15%). This 
may be the result of their level of learning — this was a 
sophomore class and students at this level are not 
required nor have they developed skills in group 
dynamics and intuitive decision-making. Honey and 
Mumford indicate that individuals tend to rely on one of 
these approaches when they are engaged in learning. 
This seems to hold true here. 

Eachus’ (1993) Academic Self-Efficacy Scale assesses 
the extent to which students believe they have the ability 
to exert control over their academic environment. 
Students’ scores were all across the board here, some 
being very high and others relatively low. There does 
not seem to be a pattern with respect to gender or course 
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section. The small sample size may be a deciding factor 
and a larger sample size is needed to better understand 
the implications of self-efficacy. 

Thus, from this preliminary analysis we find that there 
are some scales that seem to better predict learning 
outcome than others. Solomon and Felder’s (1996) 
Index of Learning Styles (ILS) seems to have more 
consistent and applicable predictive value than the other 
scales. Kolb’s (1984) Learning Style Inventory also 
seems to shed some light on which learning style is more 
prevalent in distance learning. The results of this pilot 
study suggest that a larger sample might be useful in 
determining whether or not these scales do indeed 
predict learning outcomes. Correlation with students’ 
final grades might be even more of a significant factor. 
A follow-up study will be done to determine the effects 
of grade on learning style. 

IMPLICATIONS FOR EDUCATORS 

Given the sheer number of students who anticipate 
taking or are currently taking distance-learning courses, 
the need to be able to quickly and easily assess their 
potential level of success in these courses is paramount. 
Simply because it is an available alternative to the 
traditional classroom does not make it a viable option for 
everyone. Students have specialized needs and skills 
and not every student may be suited to a distance- 
learning environment. It is the responsibility of 
educators to make sure that students know and 
understand the risks and potential drawbacks to this 
environment. The last thing we want is for a student to 
be “lost in cyberspace” when a simple assessment early 
on might have identified the student as a poor candidate 
for distance learning. Regardless of the attractiveness 
and profitability of this new pedagogy, we must still be 
available for our students and provide them with every 
opportunity to further their education as they go forward 
on their journey in today’s fast paced digital world. 
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WHAT MAKES DISTRIBUTED 
LEARNING EFFECTIVE? 



Panel 

Panelists (in alphabetical order): 

Thomas L. Case 
Georgia South ern Un iversity 

Diane Fischer 
Dowling College 

Muhammadou M. O. Kah 
Rutgers University 

Carol Okolica . 
Dowling College 

Raymond Papp 
University of Tampa 



DESCRIPTION 

Technological advances in hardware, software and 
networks have made it possible to move the contents of 
a course completely on-line and new tools such as 
threaded discussion groups, chat rooms, and virtual 
lectures allow professors to conduct a class entirely on 
line. These new tools are available from a number of 
commercial providers and are necessary for the 
development and management of a distance-learning 
course . 

Effective use of distance learning technologies in the 
classroom can transform the learning pro cess. The use of 
higher-order skills such as problem solving, 
collaboration, statistical analysis and simulation 
enhances student learning. Assigned projects require 
greater student initiative and enable them to incorporate 



“real world” scenarios to supplement traditional 
learning. Current distributed learning tools provide 
instructors with powerful tools to monitor, guide and 
assess the progress of their students as well as the ability 
to bring subject matter experts into the classroom 
virtually. Thus, these learning information systems can 
be used to track student performance in real time and 
overtime. Instructors can also use distributed learning 
tools to access resources to supplement instruction and 
exchange ideas with other instructors and professional 
experts in their domain. Such learning tools have the 
potential to become not only an “instructors aide” in the 
classroom, but a complete learning information system. 
Characteristics of effective distributed learning include: 

• High Expectations 

• Effective Instructors 

• Technology in every students hand 
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• Engaging and interactive high quality content 

• Internet connection 

• Adequate learning time 

A pre-requisite to a successful course in a distributed 
learningmode is content that sets high expectations from 
students through effective delivery of challenging 
subjectmatter in a manner that is motivating to students. 
It is also important to carry content on user-friendly 
pi at forms that utilize multimedia tools that students tend 
to be attracted to. Finally, instructors delivering course 
content must be able to effectively use the technologies. 
This might require them to modify or shift their 
pedagogical paradig m and behav ior. The myriad benefits 
of integrating distributed learning technologies into the 
classroom when successful include: 

• Student Motivation 

• Student Achievement 

• Higher level thinking 

• Gives instructors tools to improve instruction 

• Utilizes resources of the WWW 

• Expands learning time 

• Prepares students for the digital world 

Our experiences in developing distance-learning courses 
date back to as far as 1996. As early adopters of the 



Internet and World Wide W eb (WW W) in our traditional 
on-campus classes, it seemed to be a logical and natural 
extension of what we were already doing. Much of our 
course material was already on-line and accessible 
through our course pages or through Blackboard™, 
eCollege™, WebCT™ or some other provider. Many of 
us had detailed web pages and external links to other 
resources on-line and students were quite pleased with 
the amount of supplemental information provided to 
them. Our shared dilemm a was our uncertainty of how 
to duplicate the “classroom environment” in an on-line 
setting and how students would respond to the lack of 
face-to-face, personal contact typical of an on-campus 
course. 

PANEL FORMAT 

The members of this panel will share their experiences 
with distance learning and focus on the challenges they 
face and how they have solved/attempted to solve these 
challenges. Focus will be on what each of us has found 
to be most effective in teaching in a distance-learning 
environment. Each member will share their own 
experiences and then the panel members will field 
questions from the audience. 
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ALPHA IOTA MU: IMPLEMENTING AN 
INFORMATION SYSTEMS HONOR SOCIETY 



James R. Buffington 
Indiana State University 

Brent L. Bacon 

Alpha Iota Mu International Fraternity , Student President 



ABSTRACT 

Honor societies provide a valuable function in service to professional associations and prov ide important benefits 
to members . Alpha Iota Mu, under the auspices of the International Academy for Information Management, was 
formed as the honor society for information systems students and professionals. In this paper, we describe how 
an institution can form a chapter of Alpha Iota Mu. 



THE ROLE OF HONOR SOCIETIES 

Honor societies have long been established, parti- 
cularly in American academic culture, as a means of 
promoting high standards in a specific discipline and 
as a means to honor those who are especially de- 
serving. The Association of College Honor Societies 
(ACHS), founded in 1925, is the nation’s only 
certifying agency for college and university honor 
societies. Currently, the ACHS certifies 65 honor 
societies. One of those societies is UpsilonPi Epsilon, 
the national scholastic honorary society for students of 
computer sciences. However, there is no corre- 
sponding honor society for students of information 
systems. The authors’ goal is to establish Alpha Iota 
Mu as the certified honor society for information 
systems. 

THE HISTORY OF ALPHA IOTA MU 

Alpha Iota Mu was founded under the auspices of the 
International Academy of Information Management 
(IAIM). In 1996, President Sheila Pechinski and the 
Board of Directors of IAIM authorized Indiana State 
University to found the Alpha chapter of AIM . 

The first induction was held at Indiana State in May 
1997. Sixteen students, four faculty, and one alumnus 



were honored. Spring induction ceremonies have been 
conducted since that time. In 1999, Johns Hopkins 
University became the Beta Chapter, and Southern 
AlabamatheGammaChapter. Georgia Southern came 
aboard in 2000 as the Delta Chapter. Currently, four 
schools are in the initial stage of forming chap- 
ters — George Washington, University of Tampa, 
Mary mount, and Delta State. 

Since its inception, the stated purpose of Alpha Iota 
Mu is to foster academic excellence, to promote high 
ethical standards among information systems 
professionals, and to encourage the growth of the 
profession. 

BENEFITS OF MEMBERSHIP 

All honor societies confer benefits to their member- 
ship, Foremost among these benefits is the lifetime 
recognition provided by membership. As time passes 
and the prestige of the society grows, employers 
increasingly realize the value of members’ academic 
achievements. Membership can also assist in advan- 
cing the career opportunities of members. As members 
graduate and assume important positions in the 
business community, they provide excellent sources of 
help and guidance for future employment. The oppor- 
tunity Alpha Iota Mu student and alumni members to 
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network with business professionals can provide 
members with a significant advantage in competition 
for employment. 

Ultimately, Alpha Iota Mu plans to provide other 
services, including annual newsletters, leadership 
forums, on-line learning facilities, and scholarships. 
As Alpha Iota Mu matures, we will be able to provide 
on-line community services similar to those provided 
by Beta Gamma Sigma (Beta Gamma Sigma, 2001), 
which includes members-only features that add value 
to the BGS lifetime membership. These features range 
from free web space, to email address and forwarding, 
to searchable on-line directory, to message boards and 
on-line chat rooms. Beta Gamma Sigma’s site allows 
members to participate in timely and thought- 
provoking polls and surveys on current business 
topics. 

STARTING A CHAPTER 

The purpose of this paper is not only to inform 
educators about Alpha Iota Mu and not only to 
encourage the formation of chapters but also to 
provide specific details about starting a chapter. While 
fully realizing the advantages of installing a local 
chapter, many dedicated academics are wary of the 
commitment of their time and their energy to the 
installation and maintenance of a local honor society. 

The authors are dedicated to the principle ^ of 
streamlining the acceptance procedure, providing 
guidelines which minimize the time and effort required 
to institute a local chapter. We have learned from our 
previous experience and our improving our process. 

Faculty who are interested in starting a local chapter 
should visit the Alpha Iota Mu website at 
http://alphaiotamu.org/. Brent Bacon, president of the 
local chapter and Webmaster, deployed the website in 
September, 2001. The website includes instructions 
for forming a chapter. The first step is to form a 
committee — interested faculty, students, alumni, com- 
munity leaders, business or IS advisory boards — all 
can make valuable contributions and suggestions. 
However, it is our recommendation that the sponsor 
appoint a small steering committee to make the few 
key decisions that have to be made — requirements for 
induction, e.g. At Indiana State, the committee which 
formed the Alpha Chapter consisted of one faculty 
advisor and three academically gifted students. Our 
MIS Advisory Board also offered input and advice. 



The steering committee should draft local chapter by- 
laws and submit them to the international governing 
board at sdbuff@befac.indstate.edu for approval. To 
assist local chapter in the drafting of by-laws, we 
provide a template — the by-laws which have been 
adopted for use at Indiana State. Because each 
institution is to some degree unique, it is expected that 
each chapter’s by-laws will differ to reflect those 
unique features. For example, there will always be 
minimum academic and character standards required 
for induction. Those standards can vary from school 
to school, so long as they are approved by the 
governing body. Once the chapter by-laws have been 
approved, or approved as amended, the chapter is 
authorized to conduct induction ceremonies. 

There is a $50 fee for chartering the local chapter. The 
local chapter will then be provided with a chapter 
charter and the oath of induction. In order to conduct 
the initial ceremony, a member in good standing 
should administer the oath. The governing body can 
assist in the induction of the initial member. 

The governing body can provide you with other 
guidelines, too. The governing body can provide 
templates for scripting the induction ceremony, audio 
pronunciation of the Greek symbols, guidelines for 
speakers, etc. Pins bearing the Alpha Iota Mu logo 
and certificates for individual members are available 
for $18 each, plus $15 shipping and handling. 

SUMMARY 

We believe that forming a chapter of Alpha Iota Mu, 
the honor society for students of information systems, 
performs a valuable function to the profession and to 
the members. We have formulated a procedure to 
minimize the expenditure of time and energy in 
chapter formation. We encourage all interested indivi- 
duals to contact us to initiate chapter formation. We 
believe the intrinsic rewards alone are worth the costs. 
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ABSTRACT 

In many MIS or business computing programs , students are ready to take on larger projects in intermediate 
programming classes. These classes are often taken before students are exposed to databases , systems analysis and 
design, and user interface design. As a result, even simple business projects can be difficult to manage. 

One alternative is to have the students build simple games. Games encourage experimentation with animation, 
sound, and other special effects. At the same time, more typical requirements can still be included such as file input 
and output, standard algorithms (sorts and searches), and reports (game summaries). More importantly, games can 
be more fun to build than typical business projects. This should increase the amount of time spent on the project, 
and thus students should learn more. Game projects should also encourage creativity and experimentation. 

This study compared 2 5 game and 22 business projects written in Visual Basic under similar conditions. The game 
projects observed in this study turned out to be almost twice as large and more than twice as complex as the business 
projects, using common software size metrics. This suggests that students are spending more time writing them and 
presumably learning more. 



INTRODUCTION 

In many MIS or business computing programs, students 
are ready to take on larger projects in the second or third 
programming classes. These classes are often taken 
early in their programs, so students have only limited 
exposure to databases, systems analysis and design, and 
user interface design. As a result, these students can 
have difficulty managing even relatively simple 
business projects. 

One alternative that addresses some of these concerns is 
to have the students build simple games. Dobing and 
Erbach (1999) noted that such projects offer “clear 
goals, room for creativity (particularly in the user 
interface), and scalability (partial implementations are 



still playable). Games encourage exper imentation with 
simple animation, sound, and other special effects. But 
at the same time, game project requirements can still 
include more traditional features such as file input and 
output (e.g., game logs and saving), standard algorithms 
(sorts and searches), and reports (game summaries). 
The paper concludes with the following claim. 

While we cannot quantify the results, we 
believe that students spent more time on the 
game projects than they would have with 
traditional projects, exhibited greater creativity, 
and experimented mere with aspects not taught 
in class (particularly Visual Basic controls). 
Thus, we believe they probably learned more 
and that is after all the primary goal of the 
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project and the class (Dobing & Erbach, 1999, 

p.302). 

This study provides some empirical evidence to partially 
support these claims. The game projects observed in 
this study turned out to be almost twice as large and 
more than twice as complex as the business projects, 
using common software size metrics. However, there 
was no evidence of greater experimentation with 
controls and events in the game projects. Creativity is 
difficult to define in this context and was not included in 
the study. 

Perhaps the most interesting issue is determining how 
the effectiveness of different assignment types should be 
assessed. Instructors commonly grade assignments 
against each other and against some expected standard 
to arrive at a quality measure (grade). But what criteria 
should be applied when comparing different 
assignments to determine their relative effectiveness at 
helping students achieve their learning goals? 
Conventional software metrics are suggested as a useful 
and relatively simple measure for this purpose. 

RESEARCH HYPOTHESES 

Based on the educational literature, we expected 
students would spend more time on game projects. 
While we could have asked students to log the time 
spent on their projects, the reliability of such self- 
reported measures seems low based on anecdotal reports 
from students involved in previous studies that used this 
method. Thus, we instead chose to study the size of the 
output rather than the effort: 

HI: Students working on game projects will write larger 
and more complex programs. 

Because students were expected to spend more time on 
game projects, we also expected they would experiment 
more with different features of the language. Visual 
Basic has a large library of controls (with associated 
events). There is both a basic toolbox of controls that is 
normally displayed on the screen when a new project is 
opened, plus an extensive additional set that can be 
added. These additional controls have their own 
associated events. Greater experimentation should mean 
that: 

H2: Students working on game projects will use a wider 
variety of interface controls and events. 



Ideally, we would like to test a third hypothesis, that 
students who work on game projects learn more and 
produce projects of higher quality. Both of these are 
very difficult to measure, with software quality perhaps 
being the best evidence of learning. Software quality is 
most commonly measured through user satisfaction and 
defect counts (Jones 1991, p. 22). Grades are similar to 
a satisfaction measure, but are obviously subject to bias 
(particularly across different projects) and reliability 
concerns. Using multiple graders could improve 
reliability, but the resources required would be 
considerable. Quality must also consider the complexity 
of the task. Which has greater merit, a highly simplified 
game that works well ora more complex one with some 
defects? Can the code used to move Monopoly pieces 
be meaningfully compared to code used to slot courses 
into a program or select an automobile for purchase? 
These questions are not easily answered. 

Defects could include run-time errors, inaccurate 
responses, and missing features and all are normally 
reflected in the grades. But there are problems 
measuring them in a reliable and unbiased way. Run- 
time errors are observable, but often depend on the order 
of the events (which, in games, can be random). Errors 
can also prevent testing subsequent code (unless fixed 
by the instructor). Accuracy can be verified, but 
different types of projects offer quite different levels of 
difficulty. Identifying missing features and crediting 
extra ones also becomes more complicated when 
students are encouraged to add variations to games or 
other tasks, as was done in this study. 

Maintainability is another dimension of quality, and 
within that, adherence to coding standards would be the 
easiest aspect to measure. However, coding standards 
changed with the textbook used, different Microsoft 
recommendations and instructor preferences during the 
experimental period. Other measures of maintainability, 
including module cohesion and coupling, are even more 
difficult to apply. 

Because of these difficulties, the only currently 
available measure of quality is the grade assigned by the 
instructor. Further research to identify better approaches 
would be welcome. 

EXPERIMENTAL DESIGN 

The experiment was run at a smaller university (about 
6,000 students) that offers separate Computing Science 
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and MIS programs. All MIS students must take the 
introductory Computer Science course, which was 
taught in Pascal, Java or C++ over the course of this 
experiment. Neither Visual Basic, nor any othervariety 
of Basic, was ever used for that course or any other 
Computer Science course. The MIS program offers its 
own required course in Visual Basic, which was used for 
this experiment. 

Some of the Visual Basic students have more 
programming courses and experience than just the 
Computer Science prerequisite mentioned. The MIS 
program has a second required programming course that 
was taught in either C++ or Java during the period of the 
experiment. This course emphasizes object-oriented 
programming while the Visual Basic course focuses on 
the visual programming environment. The courses can 
be taken in either order. Some students also have 
related work experience, either through the University’s 
Coop program or jobs they find on their own. However, 
only a few students had ever worked with Visual Basic 
previously and none had the equivalent of an 
introductory course. 

There are also a few Computer Science students who 
take the course each term. Some are doing combined 
MIS/Computer Science degrees. These students have 
been included in the study. There have also been four 
graduates who have taken the course, either as part of 
their ongoing job training or towards agraduate degree. 
Projects done by these students, always done 
individually or with another graduate, have been 
excluded from the analysis. 

The same instructor taught all Visual Basic classes 
offered in the MIS program during the experimental 
period. The course content and the students were 
reasonably consistent from term to tenn. The instructor 
first taught Visual Basic (Version 5) in fall 1998 as a 
special elective course using Zak (1998) as the text. 
Because the fall 1998 term was the first VB course 
taught, the software was a different version, larger team 
sizes were permitted, and students could choose their 
own projects, the results are not included in the analysis. 



Based on positive student response, the course was 
added to the MIS program requirements. While the 
software was changed to VB 6 in spring 1999, the 
textbook remained the same and course was still 
essentially taught as VB 5. By fall 1999, a Visual Basic 
6 textbook was adopted (Bradley & Millspaugh, 1999) 
and used through Spring 2001. The full Professional 
Versions of Visual Basic were available in the labs for 
all classes, and students also had access to the more 
limited versions that accompany both textbooks. The 
class is taught in a lab where every student has a 
computer, with an enrollment limit of 23. One class was 
taught in each term except for spring 2001, when 
demand was sufficient to offer two section^. Each class 
completed four smaller assignments (worth 5% each) 
before beginning their projects. The project was worth 
30 or 35% of the final grade for each of the six classes 
taught over five terms. 

In all classes used in this study, students chose their own 
partners. Only two-person teams were permitted to 
control the “free rider problem. Some chose not to have 
a partner, could not find a partner, or were abandoned by 
their partner part way through the project. There were 
only a few single-person projects and they tended to be 
smaller, so they are not included in the analysis. 

Each term, students were asked to suggest projects and 
then vote for their preference. There is no apparent 
reason why students switched from preferring games to 
business projects in Fall 2000. Some students in both 
Fall 2000 and Spring 2001 were disappointed because 
they had taken the course with a game project in mind. 
The change was not imposed to create data for this 
study. The professor actually encouraged selection of a 
game project in Fall 2000 and was surprised by the vote. 
Nor is there any obvious reason to explain why more 
students chose to work individually on the business 
projects. Students are told that all projects are graded 
equally, regardless of whether there are one or two 
programmers. Table 1 summarizes the projects used 
and numbers of teams (e.g., spring 1999 had 9 two- 
person and 2 one-person projects). 
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TABLE 1 

PROJECTS AND TEAMS 



Term 


No. of 
Projects 


Project Type 


Spring 1999 


9 (2), 2(1) 


Monopoly 


Fall 1999 


9 (2), 0 (1) 


Trivial Pursuit 


Spring 2000 


9 (2), 1(1) 


Yahtzee 


Fall 2000 


5 (2), 4(1) 


Automobile Dealer 
Vehicle Purchase 


Spring 2001 


17 (2), 5(1) 


MIS Student 
Program Planner 



The Monopoly, Trivial Pursuit and Yahtzee projects 
required conversion of the standard board games to a 
computerized format. Students were encouraged to 
customize the formats. For example, two Monopoly 
games used local street names and another was based on 
local bars. The Yahtzee project offered less opportunity 
for customization, although there were both Sesame 
Street and Spice Girls versions. The Trivial Pursuit 
game was based on a school class, with questions built 
around the student’s favorite sports, foods, songs, etc. 
The program could then generate random multiple- 
choice questions, e.g., “What was Chris Smith’s favorite 
song? The incorrect answers came from other 
students’ responses. 

One disadvantage of game proj ects i s that there arc often 
Visual Basic versions (complete with source code) 
available on the Internet. There were apparently no VB 
Monopoly versions available when that project was 
used. In contrast, several version of Yahtzee could 
easily be found. To limit their usefulness, some 
additional requirements were imposed that were not met 
by the Internet versions. Similarly, someTrivial Pursuit 
games are available but the required version was quite 
different and most students created their own board 
layout. 

The first business project, the automobile dealer vehicle 
purchase system, was based on the VB Auto Center case 
in Bradley and Millspaugh (1999). Students had some 
advantage with this project because earlier assignments 
had been based on this case. For all other classes, the 
projects did not build on previous assignments. 
Moreover, the textbook provided considerable help in 
suggesting features and interface designs so students had 



a reasonably clear objective. This helped compensate 
for some of the disadvantages of business projects noted 
in the Introduction. 

The most recent “business project was an MIS Student 
Program Planner. The purpose was to help college 
transfer MIS students (the simplest type to handle) plan 
their two years of courses in the business schooL While 
no interface design was suggested, the project should 
have been easier for students to relate to than most 
business projects because they are the intended users. 
Courses taken are similar for all students and program 
planning is an exercise they have all been through. 

MEASUREMENT 

Program size and complexity were measured using three 
common approaches. The first is lines of code (LOC). 
LOC (or KLOC, thousands lines of code) has been 
frequently used to measure programmer productivity 
(e.g., KLOC per year), such as in the COCOMO model 
(Boehm, 1981) and subsequent variations. There are 
different definitions of what constitutes a LOC. Some 
include all lines; others exclude variable declarations 
and comments. We defined Lines Of Instructions (LOI) 
to include all executable statements, but exclude 
comments, blank lines, declarations, and With 
statements (but not the statements inside the With 
group). Non-Commented Lines Of Code (NCLOC) 
were defined as any line of program text that is not a 
comment or blank line, which Fenton (1991) claims is 
the most common definition. While Visual Basic allows 
multiple declarations within a single line, and multiple 
statements separated by colons, this practice was not 
encouraged by the texts or instructor and was not 
commonly used. Thus, the lines of variable declarations 
should be very close to the number of variables. 
Commented Lines Of Code (CLOC) isNCLOC plus the 
number of lines of comments. The CLOC measure 
excludes blank lines and empty comments (which were 
counted as blank lines). The number of code statements 
with both code and inline comments was also recorded, 
but these comments did not affect the CLOC measure. 
Finally, Total Lines of Code (TLOC), including even 
blank lines, was also measured. 

Statements on more than one line (continuations) could 
be counted as single or multiple statements. Students 
were encouraged to use continuations whenever a line 
was too long to print without “wrapping. The number 
of continuation lines was recorded so that they could be 
included in the NCLOC and CLOC measures if desired. 
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Because continuations within comments can be used in 
lieu of repeating the comment character, comment 
continuations were treated as if they were new lines of 
comments. The same rule was applied to declarations. 

Visual Basic Form (.frm) files also include control code 
generated through the Design mode of the interface. For 
example, drawing a textbox on a screen will create 
something similar to the following lines of code: 

Begin VB.TextBox Textl 



Height 


= 285 


Left 


= 1680 


Tablndex 


= 2 


Text 


= "Textl 


Top 


= 120 


Width 


= 735 



End 

Any additional property value changed in Design mode 
results in another line of code. Adding these statements 
to the normal LOC measures would seem to 
overestimate their importance. For the projects included 
in this study, the average lines of control code (mostly 
property assignments) were almost three times larger 
than the NCLOC measure. Ignoring controls does not 
seem appropriate either; some, such as creating a 
Toolbar, require considerable time. For this study, 
separate control and code measures were used. 

The number of controls of each type used was counted, 
along with the number of unique controls (i.e., counting 
only the first instance of use for each control). The 
control count included every member of a control array. 
Menu items were also included as basic controls. 
Flowever, controls created in code were not counted 
(and were not seen in any of the students’ projects). 
Unique events were also counted, regardless of what 
type of control was involved. This is admittedly a 
limited operationalization of “experimentation. 
Counting the number of controls or functions used that 
were not required in assignments, discussed in the 
textbook, or covered during class might be better, but far 
more complicated because they vary from term to term. 
In any event, examples that meet all three criteria would 
be quite rare. 



A second type of measure is code size, measured in 
bytes. First, a File Size (FS) measure was computed 
based on the file sizes of all the code modules. Next, 
Code Size (CS) was measured as the number of bytes in 
the code statements, excluding all comments and control 
code. For this measure, all indentation was removed, 
along with extra spaces within the code. The 

Commented Code Size (CCS) measure was the same 
except that comments were included. Unlike LOC 
measures, size measures are unaffected by whether 
several statements are placed in one line or one 
statement is divided over several lines. Size measures 
have similar shortcomings to LOC measures, plus the 
additional concern that longer variable names increase 
the metric but not the coding effort. 

The third type of measure attempts to include 
complexity. The Halstead metric, as described by 
DeMarco (1982, pp. 269-272), combines length and 
vocabulary (complexity). Length is the total number of 
operators and operands (tokens in the code) and 
vocabulary is the total of unique instances of these 
tokens. Tokens are extracted using spaces, parentheses, 
periods, underscores and exclamation marks as 
separators. The formula is: 

Volume = Length * Log 2 ( Vocabulary) (1) 

This metric will be larger if the program is treated as a 
single piece of code rather than as several modules. For 
this study, the volume of each file (form or code 
module) was computed separately and the total was used 
as the Halstead metric. Comments and Visual Basic 
control code were not included in the original metric and 
are therefore excluded here as well. Character strings, 
either in code or declarations, are treated as a single 
token regardless of length and internal spacing. 

The other complexity measure used was based on 
McCabe’sCyclomaticMeasure. DeMarco (1982, pp. 1 19- 
122) provides a simplified version for a module (equi- 
valent to a Visual Basic procedure) that is a count of all 
decision statements, including loops, plus one. To get a 
project measure, the count of all decision statements is 
added to the number of procedures. This is referred to 
here as the Decision Point (DP) metric. Both the original 
McCabe measure and the simplified version used here are 
unaffected by control code, which contains no decision 
points. A more recent and detailed presentation of the 
McCabe measure can be found in Watson and McCabe 
(1996). 
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All three types of software metrics have been rightly 
criticized as management tools for several reasons. 
They are difficult to compare across environments, 
which may have very different typical code density. 
They do not accurately reflect the implications of such 
decisions as the unrolling of loops to squeeze out alittle 
faster performance. In addition, employees who know 
they are being assessed in this way can manipulate them. 
They are also less effective as predictors of 
implementation time and effort; function points are 
often preferred for that task (Jones, 1991). 
Nevertheless, as Fenton and Neil (1998) point out, the 
simple metrics remain in use precisely because “they are 
easy to understand and simple to collect. They also 
note that LOC measures are “routinely used and the 
McCabe metric “remains exceptionally popular. 

More importantly, for the purposes of this study, many 
of the concerns raised do not apply. The students’ 
projects were of comparable complexity, all using the 
same version of the same language. All students 
followed the same software standards (as set out in the 
textbook and classes) and, in a sense, reported to the 
same manager (course instructor). Projects were judged 
on quality, not on their size. Indeed, the students did not 
know that the projects would also be measured in this 
way. Function points vary across individual projects 
because some teams attempt more complex 
implementations. Consequently, it is a reasonable 
expectation that the average size of the code base of 
these projects is in fact representative of the effort that 
students put into them. 

For all three measures, code in “.bas (standard or code 
modules), “.els (class modules) and “.frm files (form 
modules) was counted. There can also be code inside 
ActiveX controls (“.ctl files), but only one ActiveX 
control was used in the student programs and that was a 
copy of a class example (a digital clock, which was 
irrelevant to the project). Therefore, ActiveX code was 
not included in the measures. Visual Basic classes were 
included, but only two projects contained them, so no 
separate count of this code was made. The code for 
automatically generated About Forms was excluded 
from all measures, although it was included when the 
students apparently generated their own code. (The 
distinction was based on some of the comments included 
in the automatically generated code.) Regardless of how 
it was generated, the About Form was included in the 
Form count. Finally, the code in the automatically 
generated Group (.vbg) and Project (.vbp) files was not 
counted. 



A Visual Basic Code Analyzer program was written in 
Visual Basic to measure both the control and code 
parameters listed above. The number of Form, Code 
and Class Modules referenced in the project file were 
also counted, using code in the .vbp files. Counting the 
modules referenced is not the same as counting all 
submitted modules. Some projects contained extra .frm 
and .bas files not accessible through the project and 
these were ignored. However, it is possible that some 
projects contained modules and code that could never be 
executed. Determining whether a program contains 
such code is known to be a very difficult problem. 

The Code Analyzer assumes that the students’ programs 
can be compiled. Thus, it does not need to deal with 
code containing syntax errors or other problems that 
might make statements difficult to classify. For 
example, code cannot contain key words (e.g., “Do or 
“If ) as variable names and still be compiled. There 
were a few cases where programs did not meet this 
criterion, but very few statements (if any) would be 
miscounted because of it. While the Code Analyzer 
program would need further work to be applied to more 
sophisticated Visual Basic code, such as might be 
written by advanced students or professional 
programmers, the features provided are sufficient to 
analyze the code of students in their first or second 
Visual Basic class. 

RESULTS 

The analysis is based on projects completed over five 
terms (six sections) beginning in spring 1999. During 
that period, undergraduate students submitted 27 two- 
person game projects and 22 two-person business 
projects. However, two game projects contained 
substantial amounts of plagiarized code (one from 
another group and the other from the Internet). These 
projects (for which the students received a grade of 0) 
were excluded from the analysis, leaving 25. The Code 
Analyzer program was used to compute the measures 
and SPSS- 10 was used for the statistical analysis. 

The Code Analyzer provides three types of measures of 
programming effort. For Lines of Code, six measures 
can be computed. The LOI, NCLOC, CLOC and TLOC 
measures were used as defined above and summarized 
in Table 2. In addition, continuation statements were 
added into the NCLOC and CLOC measures to create 
two additional measures. However, correlations 
between the measures with and without continuation 
statements exceeded 0.99 so there is no reason to 
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include both. The measures reported in this study 
exclude continuations (i.e., a statement on more than 
one line is counted as only one statement). 

Table 2 compares the two project types using 12 
measures of size and complexity. For all measures, 
game projects are clearly larger than the business 
projects. The game projects have almost 80% more 
lines of code, with or without comments, and 90% more 
lines of instruction. The code size (in bytes) is slightly 
more than 80% larger, although file size is only 62% 
larger (because the control code is more similar in size 
across the projects). The two complexity metrics show 
the largest differences, with games being 111% more 
complex using the Halstead metric and 116% with the 
Decision Point metric. Thus, the differences are not 
only statistically significant as noted in Table 2 but are 
also much larger than had been anticipated. (The t-test 
significance levels assume that the population variances 
are not equal. This assumption generally produced 
slightly weaker significance levels than when equal 
variances were assumed.) Game projects also had an 
average of 70% more controls and 57% more control 
property statements. This is not as large a difference as 
was found in the code measures, but is still substantial. 



Thus, the results support hypothesis HI that students 
write larger programs for game projects than business- 
related projects. 

Table 3 shows some of the key metrics for each project. 
Monopoly generated the largest projects, but for eveiy 
metric the lowest average among the games (usually 
Trivial Pursuit) exceeds the highest of both business 
projects (usually the Auto Center). Individual t-test 
results are not provided because the number of projects 
in each class is too small for any meaningful analysis. 

The correlations among the size measures were very 
high. Eight of the measures in Table 2 do not include 
control code. (This excludes File Size and the three 
Control measures at the end of the table.) Their 
correlations are shown in Table 4. All are significant at 
the 0.01 level and even the weakest correlation (0.914 
between Commented Code Size and the Lines of 
Instructions) is still very strong. 

There are three pure Control Code measures (number of 
controls, number of control properties and total lines of 
control code). The last two are almost the same, so their 



TABLE 2 

AVERAGE METRIC VALUES 



Software Metric 


Game 


Business 


Size 

Diff 


T-Test 

Sig. 


LOl - Lines of Instructions 


1,021 


537 


90% 


0.00 


NCLOC Non-commented Lines of Code 


1,307 


729 


79% 


0.01 


CLOC - Commented Lines of Code 


1,558 


874 


78% 


0.01 


TLOC Total Lines of Code 


2,010 


1,233 


63% 


0.02 


FS File Size (bytes) 


189,815 


117,195 


62% 


0.03 


CS Code Size (bytes) 


41,108 


22,706 


81% 


0.01 


CCS - Commented Code Size (bytes) 


52,755 


28,835 


83% 


0.03 


Halstead Metric 


45,133 


21,421 


111% 


0.00 


DP - Decision Points 


215 


99 


117% 


0.01 


NOC - Number of Controls 


314 


184 


71% 


0.02 


NOCP - Number of Control Properties 


2,775 


1,765 


57% 


0.04 


LOCC - Lines of Control Code 


3,614 


2,316 


56% 


0.04 
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TABLE 3 

AVERAGE METRIC VALUES BY PROJECT 



Metric 


Monopoly 


Trivial Pursuit 


Yahtzee 


Auto Center 


Program Planner 


NCLOC 


1,746 


948 


1,272 


834 


698 


CLOC 


2,237 


1,079 


1,419 


1,015 


833 


cs 


60,307 


29,426 


35,050 


25,182 


21,978 


CCS 


85,177 


34,872 


40,451 


33,381 


27,498 


Halstead 


63,021 


30,050 


44,212 


23,200 


20,898 


DP 


310 


135 


211 


85 


104 


NOC 


457 


283 


204 


197 


181 


NOCP 


_ 4,089'; 


2,246 


2,054 


1,881 


1,731 



TABLE 4 

CORRELATIONS AMONG CODE MEASURES 





LOI 


NCLOC 


CLOC 


TLOC 


CS 


CCS 


Halstead 


LOI 


1.000 






' 








NCLOC 


0.995 


1.000 












CLOC 


0.964 


0.979 


1.000 










TLOC 


0.962 


0.974 


0.991 


1.000 








CS 


0.954 


0.956 


0.947 


0.952 


1.000 






CCS 


0.914 


0.927 


0.967 


0.962 


0.969 


1.000 




Halstead 


0.979 


0.981 


0.971 


0.965 


0.980 


0.955 


1.000 


DP 


0.961 


0.955 


0.933 


0.928 


0.936 


0.906 


0.960 



correlation of 1.000 is not surprising. The number of 
controls is also correlated at 0.98 with both these 
measures. The correlations among three measures that 
include Control Code (the number of controls, number 
of control code lines, and total size) and the eight size 
measures in Table 4 are provided in Table 5. These 
correlations are not quite as high (generally between 
0.70 and 0.80 forpure control code measures and around 
0.90 for total size). All correlations in Table 5 are 
statistically significant at the 0.01 level. 

The high convergence of these measures is not 
surprising. Some are inherently related by the methods 
used to compute them. However, some of the 
convergence is no doubt also due to the similarity in 
projects tasks and student backgrounds. Moreover, 
coding standards were set by the textbook and enforced 



by the instructor (beginning with the four prior 
assignments). Thus, we would not expect these 
correlations to be as high if they were measured across 
a sample of different projects from different 
organizations. 

The second hypothesis was that game projects would 
encourage a greater variety of interface controls. As 
shown in Table 2, gameprojects had 70% more controls. 
However, they used more controls of the same type 
rather than different controls. As shown in Table 6, both 
types of projects averaged just over nine different 
controls. This is slightly less than half of the 20 basic 
controls available in Visual Basic. While game projects 
did have the higher average, it is not significant and the 
difference is too small to be important. For additional 
controls, business projects had a higher average than the 
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TABLE 5 

CORRELATIONS BETWEEN CODE, 
CONTROL AND TOTAL SIZE MEASURES 





Controls 


Control Code 
Lines 


Total 

Size 


LOI 


0.768 


0.798 


0.909 


NCLOC 


0.750 


0.781 


0.904 


CLOC 


0.709 


0.751 


0.902 


TLOC 


0.713 


0.748 


0.899 


CS 


0.726 


0.751 


0.899 


CCS 


0.676 


0.725 


0.898 


Halstead 


0.713 


0.749 


0.895 


DP 


0.780 


0.821 


0.924 



TABLE 6 

DIFFERENT CONTROLS AND EVENTS 



Metric 


Game 


Business 


T-Test 

Sig. 


Different Basic 
Controls 


9.52 


9.14 


0.51 


Different 

Additional 

Controls 


1.28 


2.68 


0.01 


Different Event 
Procedures 


5.24 


5.18 


0.92 



games. The number of different events (e.g., Click, 
Change, etc.) was also measured but proved to be 
virtually identical for the two project types. Thus, 
hypothesis H2 cannot be supported with the measures 
used in this study. 

Part of the reason that business projects have more 
additional controls is that all but one used a database. 
Only a few of the games did. For the Auto Center 
project, the database had already been set up in a 
previous assignment. The DAO data control comes 
from the basic toolbox, but the grids and data bound 
combo boxes that are often used with it come from the 
additional project components. A few teams used the 
ADO control, another additional control. Toolbars, 
which required two additional controls, were also more 
common on the business projects. 



As discussed earlier, comparing the average quality of 
projects in one assignment to another is not easy and 
grades may not be comparable from one assignment to 
another. In this case, the game projects averaged 78% 
while the business projects averaged only 69%, 
significant at the 0.02 level. However, grades are a 
much more subjective measure and fully controlled by 
the researcher so this evidence is hardly conclusive. As 
a general observation, it would seem that the average 
quality of game projects at least matched and probably 
slightly exceeded that of the business projects. Thus, 
the real difference was in the relative sizes. 

In general, project grades correlated with software 
metrics at about 0.60 (all at the 0.01 level), ranging from 
0.546 (Halstead) to 0.632 (Commented Lines of Code). 
Correlations between grades and control measures were 
about 0.35 (with significance around 0.02). The 
correlations between grades and software metrics are not 
perfect due to defects in the code, redundant code 
(which should have been collapsed into a single function 
or loop), poor user interface design, failure to adhere to 
standards, and other factors. 

CONCLUSIONS 

Based on the projects analyzed in this study, game 
projects encouraged students to write about 80% more 
code with more than double the complexity. These 
results are much stronger than we had expected. Based 
on student comments, the main reason appears to be that 
games are more fun to program. They also appear to 
engender a more competitive spirit among teams. The 
difference might have been greater had the students not 
selected business projects that were easily understood. 

However, games do not seem to encourage students to 
use a wider variety of controls or events. Most students 
relied on a fairly small set of controls that had been 
covered in class. If this type of experimentation is an 
important goal of the class, the instructor might need to 
explicitly encourage their use. For example, all game 
projects could have required a database. The Monopoly 
database could record property names and values, 
players’ worth, etc.; the Trivial Pursuit database could 
hold data for questions and answers, etc. 

In future research, we would like to broaden the measures 
used to include program quality. T ools that would also be 
helpful in grading would be particularly valuable. 
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ABSTRACT 

The choice of a language for teaching object-oriented programming is difficult. The primary goal should be to 
develop an understanding of the object-oriented paradigm in students. The language used should strongly support 
this goal. The problem is that pressures other than this goal often have an impact on the language choice. The 
objectives of teaching object-oriented programming are developed and two languages, Java and Smalltalk are 
evaluated with respect to these objectives. The analysis suggests that either language can be an effective teaching 
tool. However, more care must be taken with the use of Java to enforce the student’s understanding of object- 
orientation. 



INTRODUCTION 

Object-Oriented Analysis and Design and Object- 
Oriented Programming are becoming a very important 
subject in the information sys terns curriculum . The AIS 
Model curricula [Davis, et al., 1997] for the information 
systems major include object-oriented subjects. This 
paper focuses on object-oriented programming and 
takes the position that while object-oriented 
programming is markedly different from third- 
generation language programming, it also has some 
significant similarities. For example, students using 
either type of programming language must understand 
the core programming concepts of control structures 
(sequence, selection and iteration) and data storage and 
use (files and variables). Therefore, teaching object- 
orientation must have the primary goal of developing a 
comp lete understanding of the object-oriented paradigm 
in the student rather than just learning another 
programming language. 

The choice of programming language to be taught to 
students should include attainment of this goal as the 
primary objectiv e. However, the choice is made difficult 
by forces external to the goal. The “real world” wants 
schools to teach languages that are relevant. This 
usually means the language they primarily use in their 
organizations. Students also wantto learn programming 
skills that are marketable. These forces may put pressure 



on the institution to teach languages that are less 
desirable learning too Is than they may otherwise choose. 
Specifically, the issue boils down to choosing between 
languages used extensively by business such asC++ and 
Java and languages that are less used but may be better 
suited as a teaching tool such as Smalltalk. Additionally, 
the tool used to teach object-orientation may have a 
significant impact on the level of difficulty learning 
object-orientation experienced by students [Kolling, 
1999]. 

To better understand how the choice of language im pacts 
on the goals of teaching object-orientation, this paper 
describes the object-oriented concepts that should be 
focused on in teaching object-oriented programming. 
The paper then evaluates two languages (Java and 
Smalltalk) in terms of how well they support learning 
these object-oriented concepts. 

THE GOALS OF TEACHING OBJECTS 

The goal of teaching object-orientation with a 
programming language should focus on developing the 
student’s understanding of the object-oriented paradigm. 
Learning program ming language syntax and structure is 
very important to achieving this goal. However, there 
should be no doubt that it is a secondary goal. 
Anecdotal evidence suggests that there are many 
programmers using languages that support object- 
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oriented programming writing programs that do not 
implement the object-oriented paradigm. This is 
primarily because they do not understand it. Once a 
programmer understands the paradigm, language 
selection can be made on other factors because they can 
implement a properly designed object-oriented system 
independent of the language they learned the concepts 
in. It is merely a matter of learning different syntax 
(This may not be a trivial matter, but itis less important 
than the proper implementation of objects). 

Developing an understanding of the object-oriented 
paradigm in students involves three distinct objectives: 
(1) Developing an understanding of what an object and 
a class is, (2) Developing an understanding of how 
objects work together, and (3) Developing an 
understanding of how to use objects to implement an 
application. These goals build on one another and a 
course in object-oriented programming should begin 
with the first goal and conclude with the last Each of 
the goals is developed below. 

Object and Class 

The first objective in teaching object-oriented 
programming is to help the student develop an 
understanding of object and class. W ithin this overall 
objective are several important concepts that must be 
developed. The first concept focuses on the relationship 
between the real world and the “code world.” The thrust 
of learning is to get the student to understand how real 
world objects may be abstracted to be represented in 
program code. This also includes the importance of 
being able to program systems in such a way that the 
relationship between the system code and the real world 
it represents is very clear to individuals involved in 
developing and using the system. The second concept is 
focused on developing an understanding of the structure 
and behavior of an object. Structure is concerned with 
the data that the object maintains and behavior is 
concerned with what an object can do. The concepts of 
object and class are introduced when implementation of 
the structure and behavio r is discussed. Several items of 
importance are covered: (1) how structure is 

implemented with variables, (2) how behavior is 
implemented with methods, and (3) how the class is 
primarily a definition of an object that gives the 
programmer the ability to create actual objects that have 
the ability to maintain the required data and perform the 
required operations (behavior). The concept of 



encapsulation is introduced when discussing how only 
the object’s own methods can manipulate its data. 

Once the above two concepts have been developed, the 
student is introduced to the idea of inheritance by 
determining if the proposed object can reuse previously 
developed code. Here the student should be taught to 
examine existing objects to determine if (1) the proposed 
object is a more specialized version of an already 
existing object, or (2) if an existing object has much of 
the desired structure and behavior ofthe proposed object 
even though the proposed object is not really a 
specialization of the existing object. The ability to 
examine existing object structure and behavior is very 
important to facilitating the achievement of this 
objective. All the concepts discussed to this point are 
developed both theoretically and through example by 
actually developing the object in some code. The final 
step is to develop an understanding of how the object, 
once implemented, is used. Included is a discussion of 
object creation, message passing, and referencing an 
object. Again, this discussion should be supplemented 
with demonstration in code. 

Interacting Objects 

The second objective in teaching object-oriented 
programming is to help the student develop an 
understanding of how objects interact with one another. 
There are also several important concepts to be 
developed within this objective. The concepts developed 
in achieving this goal are highly interrelated. They 
include message passing as a way for one object to get 
services from another object, encapsulation, inheritance, 
and polymorphism. Several objects that work together 
to perform some task should be developed. The 
discussion should again include the concept of 
encapsulation. It is very important to emphasize and 
demonstrate the fact that one object cannot directly 
manipulate the data held by another object. Data is 
exchanged and changed between objects only if the 
appropriate messages are passed. The student needs to 
understand that the messages an object responds to are 
its interface. Polymorphism, the same message 
producing different responses in different objects, is 
another important concept focused on in this objective. 
It is addressed by passing the same m essage to different 
objects and showing the results. Inheritance is 
demonstrated by passing a message to an object that 
does not directly implement that method. The object’s 
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superclass implements the method and still the object 
responds appropriately. 

Object-Oriented Applications 

The third objective in teaching object-oriented 
programming is to help the student develop an 
understanding of how to use objects to implement an 
application. The primary concept within this objective 
is developing an understanding of the Model-View- 
Controller paradigm for object-oriented applications. 
The model is the functionality behind the application. 
Students should understand that this functionality is 
provided by a set of classes. The view is the interface 
to the application. This is usually a window or set of 
windows displayed on the computer screen. Students 
need to understand how to create the interface controls 
(for example, text boxes, menus, and buttons). The 
controller provides the interface between the view and 
the model. It gives function to the interface. Students 
should develop an understanding of how the view and 
model are associated withthe controller to complete the 
application. Throughout this and the previous objective, 
an understanding that students are not just creating an 
application, they are also creating the programming 
language for the application should be developed. 

EVALUATION OF OBJECT-ORIENTED 
TEACHING LANGUAGES 

Languages for teaching object-oriented programming 
should be evaluated on how well the language supports 
achieving the above three objectives. This evaluation 
was done by implementing a simple checkbook 
application Java, and Smalltalk. C++ was not included 
because it is a hybrid language that supports both object- 
oriented programming and traditional programming. 
Kolling [1999] suggests that leads to the development of 
an understanding that object-orientation is just a 
language extension rather than a completely different 
paradigm for programming. The checkbook application 
involves a checking account that has two types of 
transactions, deposits and checks. Four classes are 
required for this application: Account, Transaction, 
Deposit, and Check. Transaction is the superclass of 
Deposit and Check. It is used to demonstrate inheritance 
and reuse. A fifth class, AccountManager is also 
developed to implement the application interface. An 
outline of the structure and behavior of each class is 
identified below. 



Transaction Class 

Variables : transactionDate (date of the transaction) 

amount (amount of the transaction) 

Functions : set and retrieve the value of transactionD ate 
set and retrieve the value of amount 



Check Class (Transaction subclass) 

Variables : checkNumber (number of check posted to 

account) 

payee (receiver of the check) 

Functions : set and retrieve checkNumber 
set and retrieve value of payee 
print the check 



Deposit Class (Transaction subclass) 

Variables', type (type of deposit eg. cash, 

check) 

Functions', set and retrieve the value of type 
print the deposit 



Account Class 

Variables : accountNumber (number identifier of 

the account) 

accountName (name of account 

owner) 

balance (current account 

balance) 

transactions (collection of deposits 

and checks) 

Functions : set and retrieve value of accountNumber 
set and retrieve value of accountName 
retrieve value of balance 
(cannot directly set the value of balance) 
add a deposit to collection and update 
balance 

add a check to collection and update 
balance 

delete a deposit from collection and update 
balance 

delete a check from collection and update 
balance 



AccountManager Class 

Variables: currentAccount (account object 

displayed on screen) 

currentTransaction (transaction object 
displayed on screen) 

transactionNumber (to track the transaction 
object displayed on screen) 
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Functions', update account information from values 
entered on screen 

update transaction information from values 
entered on screen 

move forward one transaction in collection 
and display on screen 
move backward one transaction in 

collection and display on screen 
add a new transaction to account 
delete a transaction from account 




To evaluate the language, the Transaction, Check, and 
Deposit classes are implemented and tested to evaluate 
support of the first objective. Second the Account class 
is used together to demonstrate the functionality of the 
application and evaluate support for the second 
objective. Finally support for the third objective is 
evaluated by creating the AccountM anager class. 

Developing an Understanding of Object and Class 

The differences between Smalltalk and Java are 
immediately apparent in the implementation of the first 
class, Transaction. Declaring instance variables in 
Smalltalk requires no determination of access right or 
type. All variables can hold any type of object, are 
inheritedby subclasses, and are fullyencapsulated by the 
object. In Java, the developer must determine the type 
of data that the variable will hold, if the variable can be 
inherited by a subclass, and what other objects should 
have access to the variable. Smalltalk has the advantage 
here. The limited choice selection in creation of object 
variables supports the learning of object-orientation 
because all variables in all classes follow the same rules. 
Additionally, because all variables are what would be 
considered private access rights in Java, it enforces the 
concept of encapsulation in the student’s mind. 
Variables are only accessible by providing a method to 
get at the variable. Method implementation in both 
languages is similar. One difference is Java’s need to 
type variables and therefore type methods that return 
values. However, this is not a problem in learning 
object-orientation. Java’s use of access rights for 



methods can cause some confusion. All methods in 
Smalltalk can be used by other object’s to get an object 
to perform some action. In Java, the access right again 
determines who has access to the method. This requires 
the developer to pay attention to access right during 
development of the class. Both languages are equally 
adequate at helpingthe student understandthe difference 
between a class and an instance of that class (or an 
object). The instructor may create multiple new 
instances of the class and demonstrate how each has its 
own values for each variable. 

The Check and Deposit subclasses of Transaction should 
inherit the transactionDate andamount variables. While 
both languages allow this to happen, the Smalltalk class 
browser immediately shows the inheritance of these 
variables by the subclass. In Java, the variables are not 
shown in the new class definition. This can make it 
difficult for the student to understand that the variables 
are a part of the new class. Polymorphism is equally 
easy to understand with these classes. Both classes 
implement a method to print the object with the same 
name (printOn). It is easy for the student to understand 
that when a Check object is sentthe message printOn it 
will print a check while when a Deposit object is sent the 
message printOn it will print a deposit. 

Developing an Understanding of How Objects 
Work Together 

Differences between the languages are also apparent 
when developing the Account class. The Account class 
holds information about the account and all the Checks 
and Deposits made against the account. Using these 
objects in the account object begins to demonstrate how 
objects work together to accomplish some task. The first 
difference is the collection to hold Transaction objects 
(Checks and Deposits). The developer has to impo rt the 
java.util.Vector class and declare an instance variable to 
be of type Vector to hold these objects. This is 
essentially the same thing the student saw when creating 
a Date data type in the Transaction object. Differences 
in the implementation of polymorphism become more 
apparent here. The first difference between the 
languages is in implementing a method to add a 
Transaction object to the collection. This method needs 
to add the transaction and eitherincrement or decrement 
the account balance depending on whether a Check or 
Deposit has been added. Java has the advantage because 
it can use polymorphism in the fonn of method 
overloading to implement this functionality. In Sm all- 
talk, the developer has to explicitly determine the type of 
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object being added. This is easiest to see by comparing 
actual code. 

Java 

public void add(Check c) //method name 

{ 

balance = balance - c.amountQ; //get check amount 
and subtract from balance 

transactions. addElem ent(c); //add check to collection 

} 

public void add(Deposit d) //method name 

{ 

balance = balance + d.amoun t(); //get deposit amount 
and add to balance 

transactions. addElem ent(d);//add deposit to 
collection 
} 

Smalltalk 

add: aTransaction “method name” 
(aTransactionisCheck) “determine if object isa Check” 
iff rue: 

[ 

balance = balance - aTransaction amount, 
“get check amount and subtract” 

transactions addLast: aTransaction. 

“add check to collection” 

] 

ifFalse: 

[ 

balance = balance + aTransaction amount, 
“get deposit amount and add” 

transactions addLast: aTransaction. 

“add deposit to collection” 

] 

Two methods are written with the same name in Java. 
The only difference is the parameter. This makes the 
code easier to understand for the student because each 
method applies to only one type of object. The 
Smalltalk code has the feel of a traditional type checking 
with one exception that illustrates polymorphism: The 
message amount can be sent to the same variable and it 
will work properly regardless of the type of object it 
holds. In Java we must use variables of different types. 

In using Java to teach object orientation, it is important 
to properly protect an object’s variables with the 
selection of the most restrictive access right possible. If 
the check or deposit classes declared the variable amount 
to have anything other than a private access right the 
above methods could directly access the value rather 



than using a method to access it. While the application 
will work in this manner it violates the object-oriented 
principle of encapsulation. An object in Smalltalk 
allows access to its variables only through methods 
implemented by the object. This forces proper object- 
oriented programming because the developer cannot 
create an object that works as a set of global variables. 

The second difference is in the implementation of the 
Account class printmethod. This methodprints account 
information and a list of all transactions. Smalltalk’s 
implementation of polymorphism is better in this 
situation. The difference associated with printing of the 
set of transactions in the account. Again the code will 
help illustrate these differences. The following code is 
not complete and has been altered to help illustrate 
differences in the languages. 

Java 

public String printOn () //method name 

{ 

// printing of account information not included 

//the following code prints the transactions 
for (i=l; i <= transactions.size(); i++) 

//iterate through the collection 

{ 

if (transactions. elementAt(i) instanceof Check) 
//test if object is a check 
{ 

Check c = (Check) transactions. elementAt(i); 
//convert object to check 

c. printOn(); //print check 

} 

if (transaction s.elementAt(i) instanc eof Dep osit) 
//test if object is a deposit 
{ 

Deposit d = (Deposit) transactions. elementAt(i); 
//convert object to deposit 

d. printOn(); //print deposit 

} 



Smalltalk 

printOn: aStream “method name” 

“printing of account information not included” 

“the following code prints the transactions” 
transactionsdo: “iterate through the 
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collection” 

[:aTransaction | aTransaction printOn: aStream]. 
“print each transaction” 

The collections in both languages can hold any type of 
object. However, because of the strong typing in Java, 
the collection returns objects held in the collection as 
type Object. There is no printOn method defined in the 
object class so each object retrieved from the collection 
must be tested to determine what class of object it is. 
Once class determination has been made, the object is 
downcast , or converted, into that type of object. Then 
the printOn method may be used. In Smalltalk, it does 
not matter what type of object receives the printOn 
message. At run time the object will correctly execute 
its own printOn method. In this way, Smalltalk more 
clearly illustrates the concept of polymorphism. 

Developing an Understanding of How to Use 
Objects to Implement an Application 

Differences between the languages is minimal when 
implementing an application using the model-view- 
controller paradigm. Both languages have an 
environment for “painting” the interface, or view. Java 
has an interface that may be more familiar to students 
than the Smalltalk painter. However both provide the 
ability to create an interface using the Windows API. 
The languages are also similar in thatthey both use one 
class to implement the view and the controller. The 
view is a method in the class to create the window with 
the appropriate controls on it. The interaction between 
the view and the model is implemented with methods 
associated with control events (for example, a button 
click) and references to the model are made through 
instance variables. The controller has minor differences 
which support object-orientation in the Smalltalk 
version. In Smalltalk, all controls are distinct objects 
whose reference must be passedto each method thatuses 
them in the controller. In Java, the controls are also 
objects, however, any method in the controller can get 
access to any object in the view without explicitly 
passing an object reference. The Smalltalk imple- 



mentation has a slight advantage in enforcing object- 
oriented thinking in students. 

CONCLUSION 

Either language can be use d to effective ly teach obje ct- 
oriented programming. However, if the primary goal of 
the course is teaching objects, then more care must be 
taken in using Java than in using Smalltalk as the 
teaching vehicle. Java’s multiple access rights may be 
used improperly to violate the object-oriented principle 
of encapsulation. Polymorphism also must be planned 
for more specifically in Java than in Smalltalk. This 
becomes especially relevant when it cannot be 
determined prior to runtime what class ofobjects willbe 
used in a particular program situation. Because of this, 
the instructor must emphasize the benefits of 
polymorphism more when using Java. Inheritance also 
must be carefully planned when using Java. Access 
rights can restrict the inheritability of variables. In 
simple objects such as the ones developed for this 
evaluation, the student may not understand the benefits 
of inheritance over just creating another class if they 
constantly have to be aware of how they determine 
access rights for an object. However, these issues should 
not keep one from using Java to develop an 
understanding object-orientation. Only that extra care 
must be taken in course delivery. Java is a language 
with better corporate acceptance than Smalltalk and 
should be considered an appropriate vehicle forteaching 
objects. 
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ABSTRACT 

This paper describes a knowledge sharing/management tool that has the potential to enhance student project team 
performance in IS courses. The tool, which is called TKSS (Internet Knowledge Sharing System), provides a 
mechanism for organizing and sharing knowledge gleaned from the Internet among group members and other 
students enrolled in IS courses. It has the potential to facilitate the completion of student projects that include 
Internet research while simultaneously teaching users about knowledge management through hands-on exposure to 
a working knowledge sharing tool. 



INTRODUCTION 

It is widely recognized that in order to dynamically 
respond to a changing environment, an organization must 
do more than process information efficiently. It must also 
create information and knowledge (Nonaka, 1994). 
Companies that effectively create and process knowledge 
can leverage this ability to achieve a competitive strategic 
advantage. 

Student teams in IS courses are often called upon to 
function as mini-organizations whose success is directly 
linked to their ability to acquire, manage, share, and 
process knowledge. This is especially true when student 
teams are challenged to assemble, filter, evaluate and 
synthesize research on IS topics in order to develop group 
deliverables on emerging technologies and other IS topics. 
The information and knowledge management challenges 
faced by student teams are amplified when group 
assignments require the acquisition and synthesis of 



information and knowledge from the Internet’s vast pool 
of resources. 

Increasingly, information systems educators are requiring 
student teams to complete group assignments that involve 
Internet-based research and/or require student-to-student 
collaborative interaction via the Internet (e.g., 
Bialaszewski, Case, and Wood, 1996; Buffington & 
Wood, 1997; Chimi and Gordon, 1997). IS educators 
often use such assignments to illustrate the challenges of 
information and knowledge sharing via the Internet. These 
types of assignments also serve to provide students with 
first-hand insights into distributed collaboration 
(Munkvold and Line, 2000) and virtual team coordination 
and management (Sarker, Lau, and Sahay, 2000). 

Numerous researchers have recognized long-term benefits 
of instructional approaches designed to help students 
learn to be effective contributors to team projects (e.g., 
Alavi, Wheeler, and Valacich, 1995; Fellers, 1996; Van 
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Slyke, Trimmer, and Kittner, 1 997). Other researchers 
have illustrated the value of employing groupware (e.g., 
de Vreede, Briggs, and Santanen, 1999), computer 
supported collaborative learning (CSCL) 
mechanisms/tools (e.g., Brandon and Hollings-head, 
1999; McConnell, 1994; Seufert and Seufert, 1998) and 
computer support for distributed collaborative learning 
(CSdCL) (Fjuk, 1998) as tools for facilitating 
collaborative learning and team-based 
information/knowledge creation and sharing. 

Dun and Hackney (2000) argue that it is time to move 
traditional approaches for facilitating collaborative 
learning among team members. They suggest that student 
exposure to student-oriented intranets and other 
knowledge sharing tools should be incorporated into IS 
curricula. Their rationale derives from mounting evidence 
that knowledge management concepts must mastered by 
students to survive in contemporary businesses that 
increasingly demand continual learning (DeGues, 1997). 
These arguments are also bolstered by an investigation 
conducted by Endlar (2000) demonstrating that intranets 
can be effectively leveraged to support team learning. 
They are also supported by the burgeoning body of 
research indicating that knowledge sharing is a key aspect 
of knowledge management processes in today’s 
organizations. 

The rationale underlying the framework on which I-KSS 
is based is outlined in the following section. Subsequent 
sections describe a conceptual framework for Internet 
knowledge capture, storage, and sharing and a description 
of a software application derived from the conceptual 
framework: I-KSS. Then, the paper presents potential 
uses of I-KSS in IS courses. 

KNOWLEDGE SHARING 
IN TODAY’S ORGANIZATION 

Numerous studies have explored how organizations 
create, store, and disseminate knowledge (e.g., Grant, 
1996; Kogut and Zander, 1992; Teece, Pisano, and 
Shuen, 1997). Knowledge in organizations is generated 
through collaboration, interaction, and relations among 
groups or units. The knowledge is stored and integrated 
to form organizational knowledge, such as competence, 
capability, or routines that can be used for creating and 
sustaining competitive advantages. Similar knowledge 
management activities are required of student teams in IS 
courses. Like groups/teams in business organizations, 



student teams are unlikely to fully realize the benefits of 
the information/knowledge that they acquire through 
completing group assignments unless the knowledge is 
effectively shared. 

Knowledge sharing has been identified in many studies as 
a key knowledge management challenge (e.g., Duffy, 
2001; Dyer and Nobeoka, 2000, Gupta and 
Govindarajan, 2000). To address knowledge sharing 
challenges, a number of organizations including Chevron, 
Xerox, Johnson & Johnson, Ford, and Whirlpool, have 
encouraged the creation of knowledge-intensive cultures 
that promote knowledge sharing (Stepanek and Brown, 
2000 ). 

Increasingly, organizations are tapping the vast pool of 
information resources available on the Internet for 
information/knowledge creation and sharing. The Internet 
has grown to encompass diverse information resources 
including trade publications, product and service 
catalogs, and government information. Many 
organizations including General Electric, IBM, Merrill 
Lynch, and Xerox, are leveraging the Internet’s vast pool 
of information for strategic planning (Pawar and Sharda, 
1997). The Internet is also being used to capture 
information about competitors, including company 
profiles and industry analyses, that is subsequently 
shared, published, stored and leveraged for strategic 
decision processes (Leibs, 1998). 

Information technology is widely recognized as an 
important enabler of knowledge acquisition and 
dissemination (Hansen, Nohria, and Tierney, 1999). 
Other studies suggest that Internet tools, such as e-mail, 
search and retrieval tools, and information repositories, 
are potential facilitators of knowledge sharing and 
integration within organizations (e.g., Alavi and Leidner, 
1999; O’Leary, 1998). 

I-KSS: 

AN INTERNET 

KNOWLEDGE SHARING SYSTEM 

The Internet Knowledge Sharing System (I-KSS) 
leverages the Internet and information technologies, such 
e-mail and knowledge repositories, to facilitate sharing 
information among the members of groups within 
organizations. The tool itself derived from a conceptual 
framework that can be traced back to fundamental 
knowledge management concepts. 
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Nonaka (1994) has identified two types of knowledge: 
tacit and explicit. Tacit knowledge is personal knowledge 
embedded in individual experience. Explicit knowledge is 
formal knowledge that can be found in an organization’s 
documents. Unlike tacit knowledge, explicit knowledge is 
relatively easy to transfer and share. Organizational 
knowledge can be created through processes that integrate 
and leverage tacit and explicit knowledge. Alavi and 
Leidner (1999) define knowledge management as a 
systematic, organizationally specified process for 
acquiring, organizing, and communicating the 
organization ’s explicit knowledge and the tacit knowledge 
of employees in ways that enable others to be more 
effective and productive. 

The knowledge management process involves three major 
sets of knowledge activities: knowledge generation, 
codification, and utilization (e.g., Alavi, 2000; Davenport 
and Prusak, 1998). Organizational knowledge is 
generated through collaboration, interactions, and 
relations among groups or units. Organizations codify 
knowledge into accessible and applicable formats for 
reuse (Davenport and Prusak, 1998). The codification of 
tacit knowledge presents special challenges to knowledge 
managers (Hansen et al., 1999). As noted by Alavi 
(2000), knowledge generation and codification are only 
beneficial in organizations that are also effective in 
knowledge utilization (the sharing, integration, and 
application of the specialized knowledge of their 
employees). 

Several studies have indicated that knowledge sharing can 
contribute to competitive strategic advantage (e.g., 
McEvily, Das, and McCabe, 2000; Teece, 1998). These 
studies indicate that effective knowledge sharing enables 
organizations to improve efficiency, reduce training cost, 
and reduce risks due to uncertainty. 

Several investigators have proposed alternative ways to 
promote knowledge sharing within organizations (e.g., 
Gupta and Govindarajan, 2000; Hansen et al., 1999; Lei, 
Slocum, and Pitts, 1999). They suggest that knowledge 
sharing can be enhanced by: 

• Using rewards or incentives: Employees in 

organizations often have little incentive to share their 
own knowledge and insights with others (Lei et al., 
1999). In such instances, it may be necessary to 
motivate their willingness to share through the use of 
explicit rewards or incentives. Establishing an 



appropriate knowledge sharing reward/incentive 
structure can also help organizations create knowledge 
sharing cultures. 

• Using effective processes to codify knowledge : It is 
recognized that tacit knowledge cannot be easily 
codified due to the nature of this type of knowledge. 
Explicit knowledge less challenging to codified and 
store in files and documents (Hansen et al., 1999). 
Because the ease with which decision makers can 
access and leverage codified knowledge is an important 
determinant of the utilization and effectiveness of a 
knowledge management system, it is important to 
ensure that the organization’s explicit knowledge is 
appropriately codified. 

• Using different sharing mechanisms for explicit and tacit 
knowledge : Document exchange is highly efficient and 
effective to share codified knowledge, whereas personal 
contact is likely to be more effective for sharing tacit 
knowledge (Gupta and Govindarajan, 2000). 

• Using groupware to facilitate communication among 
team members: Groupware has often been viewed as 
have the potential to promote knowledge sharing by 
capturing knowledge and enabling communication 
across groups or units. However, groupware’s 
limitations, including technical complexity and poor 
connections with Internet-based tools, restrict 
groupware’s potential to facilitate knowledge creation 
and sharing within organizations (Dennis, Pootheri, and 
Natarajan, 1998). 

• Leveraging data warehouses for knowledge 
management: Data warehousing has also been 
recognized as having numerous knowledge sharing 
benefits including making accurate information 
available to decision makers and providing better 
access to information. Davenport, De Long, and Beers 
(1998), for example, emphasize the significance of 
knowledge repositories in knowledge management 
initiatives and note the key role that they may play in 
promoting knowledge access, and knowledge sharing. 

• Using the Internet to expand the organization ’s 
explicit knowledge: The Internet’s vast pool of 
information resources is being increasingly used by 
organizations to support information and knowledge 
creation and sharing. Little research attention, however, 
has been given to managing information (especially 
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unstructured information) obtained from the Internet or 
to the organizational processes for enhancing and 
sharing Internet knowledge. 

AN INTERNET KNOWLEDGE 
SHARING SYSTEM FRAMEWORK 



Figure 1 presents a framework for enhancing and sharing 
information and knowledge derived from the Internet. 
This framework is based on Choo’s framework (1995) 
for a business intelligence system. 



FIGURE 1 

FRAMEWORK FOR AN INTERNET 
KNOWLEDGE SHARING SYSTEM 
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The framework illustrated in Figure 1 assumes that 
organizations first determine a set of critical information 
needs based on their strategic objectives. Critical 
information needs vary across organizations as a function 
of differences among the nature and importance of their 
particular strategic objectives. Within an organization, 
the critical information needs associated with a particular 
strategic objective can be operationalized as a list of all 
relevant information categories and types that are 
valuable for that strategic objective. Once critical 
information needs for the organization’s strategic 
objectives have been identified, scanning experts are 
better positioned to conduct the following scanning 
activities. 

• Collection : information is gathered from external 
sources by using Internet tools including conventional 
World Wide Web search engines (e.g., Yahoo), and 
traditional on-line databases (e.g., LEXIS-NEXIS). 
Scanning experts may also be able to leverage Web 
browsers and search engines to scan Web-enabled 
internal databases and knowledge repositories. 



• Analysis : collected information is compared, collated, 
and interpreted by using scanning experts’ domain 
knowledge. One direct result of analysis is the 
immediate transfer of mission-critical or time-specific 
information to senior managers via electronic mail, 
instant messaging, or other communication 
technologies. 

• Storage : information that has been analyzed and 
determined by scanning experts to be relevant is filed 
and indexed into the knowledge repository through 
the use of add and update algorithms and information 
retrieval techniques. Domain experts can also use 
purge algorithms to ensure that the content of the 
knowledge repository does not include out-of-date 
information and/or information that has little 
relevance to the organization’s strategic objectives. 

The Internet Knowledge Sharing System (I-KSS) is 
derived from the conceptual framework illustrated in 
Figure 1. I-KSS, which is depicted in Figure 2, focuses 
on analysis and storage activities after users have 
identified relevant information from the Internet. It also 
addresses the sharing of relevant Internet-based 
information and knowledge. 

FIGURE 2 

OVERVIEW OF I-KSS’S 
KNOWLEDGE SHARING PROCESS 
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I-KSS Consists of Three Major Processes 

• Integrating relevant information with the knowledge 
repository, 

• Sending E-mail to specific decision makers if newly 
collected and analyzed information has been identified 
as mission-critical, and 
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• Organizing the knowledge repository to facilitate 
knowledge sharing. 

Each of these is briefly described below. 

Integrating Internet Information with the 
Knowledge Repository 

Once scanning experts have identified relevant 
information from the Internet, the information is 
integrated with the knowledge repository by using the 
three key operations: Add , Update , and Purge. These 
operations use distinctive keywords extracted from the 
information to determine the knowledge integration 
strategy. For keyword extraction, information retrieval 
techniques are used: stop list to eliminate common words 
(e.g., the, and, of, etc.) and stemming to identify the root 
of a word by removing or modifying prefixes and suffixes 
(Korfhage, 1997). 

In the Add and Update operations, in order to compare 
selected information with the existing information, I-KSS 
checks the elements (i.e., keywords) of Source , Subject , 
Date , and Content , in turn. Before adding Internet 
information to the knowledge repository, I-KSS first 
examines whether identical information exists in the 
knowledge repository by checking the Source. If Source 
is the same, Subject is examined next. If Subject is the 
same, I-KSS next inspects Date. Only when Date of the 
newly acquired and analyzed information is more recent 
than that of existing information in the knowledge 
repository, I-KSS adds the new information to the 
repository. In the Add and Update operations, I-KSS 
contains several actions. 

• Add the document : to save the selected information into 
the knowledge repository. 

• Update the document : to replace the existing 
information with the selected information. 

• Ignore the document', to purge the selected information. 
I-KSS observes that both the existing information in the 
knowledge repository and the selected information are 
identical because they have the same elements of 
Source and Subject. Then, I-KSS keeps the existing 
document. 

The Purge operation allows I-KSS or the user to maintain 
the knowledge repository more efficiently. Through the 
Purge operation, I-KSS is able to keep a set of 



information that matches the user’s current interests and 
reduces redundancy. I-KSS periodically examines the 
knowledge repository to keep information relevant to the 
users. If the saved information is not reviewed during a 
certain period, I-KSS moves the information from the 
knowledge repository to a temporary archive. 

Sending E-Mail 

I-KSS enables scanning experts to immediately send 
newly acquired information that is identified as mission- 
critical or time-specific to key decision makers (e.g., 
senior managers or experts) along with a reference link to 
the information. Senior managers or experts may provide 
their own opinions or comments about the new 
information and send their comments back to the scanning 
experts. I-KSS enables these comments to be included in 
the knowledge repository by binding them with the newly 
added information. I-KSS allows its users to browse 
comment information through a comment history 
interface, similar to an electronic bulletin board. 

Organizing the Knowledge Repository 

I-KSS provides several other interfaces that enable users 
to access the knowledge repository. The knowledge 
repository supports four major interfaces: search engine , 
review history, comment history, and related documents. 

• The KB search engine allows the user to search for the 
information available in the knowledge repository by 
using a list of significant keywords that indicate the 
nature of stored information contents. The search 
results include the relevancy rankings, sources, 
subjects, and partial contents of the information. 

• Review history provides the capability to browse the 
review history of specific information, for example, 
when the information was collected and analyzed, other 
users who have accessed/used the information, and 
which significant keywords were used to access the 
information. 

• Comment history provides the history of the comments 
of senior managers or experts about the specific piece 
of information stored in the knowledge repository. 
These include the date the comments were made, who 
provided the comments, and the detailed contents of the 
comments. 
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• Related Documents provides a list of other related 
information in the knowledge repository that is related 
to a specific piece of information. The similarity 
between information pieces is measured in terms of 
significant keywords. 

If the users (e.g., senior managers) have interests in a 
certain topic, they can use the KR search engine to 
retrieve the related information by selecting a single or 
multiple significant keywords available in the knowledge 
repository. The system allows the users to navigate 
among the interfaces through either the search results or 
a list of the existing information. 

I-KSS IMPLEMENTATION 

I-KSS has been implemented in Java using JDK1.3® as 
a development tool and MS Access® as a knowledge 
repository. Figure 3 presents examples of I-KSS inter- 
faces. 

The main interface has the same major capabilities as 
Web browsers such as Netscape® or IE®. Like any Web 
browser, I-KSS allows the user to navigate different Web 
sites through hyperlinks and to browse previously visited 



Web pages. The main interface also includes several 
buttons to allow users to access the knowledge repository 
and send e-mail. For example, if the user clicks the ‘Save 
& Send’ button after the information is identified as 
relevant, I-KSS stores the information into the knowledge 
repository and displays an interface for sending the 
information as an e-mail attachment. 

The KR search interface allows users to retrieve 
information from the knowledge repository by using a 
single or multiple keywords. Available keywords are 
listed in a look-up table that the user is able to access by 
selecting the ‘Look Up Keywords’ button. The look-up 
table has been arranged in the ascending order of distinct 
keywords. The search results consist of source , date , 
subject , contents , and relevancy rating. In addition, the 
interface supports Boolean searches. For example, once 
the user selects “bank”and “online” as keywords and 
chooses the ‘Any of the Words’ option, the interface 
displays a list of all of relevant documents containing 
either “bank” or “online”. With the ‘Sort By’ option, the 
retrieved information can be arranged by source or date 
in ascending order. The system also allows the user to 
explore the content of the document by selecting the 
“Open Selected File” button. 



FIGURE 3 

EXAMPLES OF I-KSS INTERFACES 
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I-KSS also allows users to navigate among the related 
knowledge repository components: for example, once the 
user selects the information from the search results and 
then chooses one of the Advanced Options , such as 
comment history, review history, or related documents, 
he/she is able to explore the selected option window. 

POTENTIAL USES OF I-KSS IN IS COURSES 

I-KSS possesses characteristics that enable it to be 
usefully employed by student teams in IS courses. 
Because it was created to be compliant with widely 
accepted GUI standards, the system is user friendly and 
can be learned quickly. Minimal training is needed to 
teach student team members to integrate newly acquired 
information from the Internet into the knowledge 
repository. Students can also easily master I-KSS ’s 
“Save and Send” features to alert fellow team members 
(or the class as a whole) that they should review the new 
assignment-relevant information that has been added to 
the I-KSS knowledge repository. The interfaces for 
supporting student queries and knowledge repository 
navigation are also easily mastered. Because of the user- 
friendly nature of I-KSS, the ability of student groups to 
assemble, synthesize, and leverage Internet information 
sources needed to develop group deliverables is enhanced. 

I-KSS has the potential to be beneficial to student teams 
in a wide variety of IS courses. It may be leveraged in 
upper level IS course that include assignments which 



require student teams to perform Internet-based research. 
This tool could be used in upper division MIS courses, 
MBA-level MIS courses, information resource 
management (IRM) courses, data communication 
courses, data management courses, and of course, 
knowledge management courses. Because I-KSS can 
support teams whose members are geographically 
distributed as well as those that are concentrated in a 
single geographical location, it has the potential to add- 
value to both distance education and traditional classes. 
Table 1 summarizes potential uses of I-KSS in various IS 
courses. 

Perhaps the most valuable outcome of allowing student 
teams to leverage I-KSS is the tool’s ability to provide 
insights about key knowledge management concepts 
during the process of completing assignments requiring 
Internet-based research. For example, in general MIS 
courses and IRM courses, I-KSS can be introduced early 
in the academic term as a tool to facilitate the assembly 
of a repository of Internet-based information and 
knowledge sources that will be needed to complete group 
deliverables on assigned topics. When attention is turned 
toward knowledge management concepts later in the 
course, student experience with I-KSS can be leveraged 
to illustrate the implementation of key knowledge 
management concepts. Because of their hands-on 
experience with I-KSS, students are likely to have a 
richer understanding and appreciation of knowledge 
management processes. 



TABLE 1 

EXAMPLES OF POTENTIAL USES OF I-KSS IN IS COURSES 



IS Course(s) 


Potential I-KSS Application(s) 


General MIS course 
(undergraduate or graduate) 


To assemble, filter, share, leverage Internet-based articles and information 
sources related to emerging technologies, e-business models or technologies, etc. 


Data Communications 


To illustrate intranets as well as Internet-oriented information and knowledge 
acquisition and sharing processes; to facilitate the completion of network design 
and implementation projects that demand students teams to assemble relevant 
product and cost information from Internet sources. 


Information Resource Management 


To illustrate the implementation of knowledge management and knowledge 
sharing processes; to facilitate the completion of students projects that leverage 
the Internet to identify appropriate measures for evaluating the effectiveness of 
specific information technologies. 


Data Management 


To illustrate the mechanics of knowledge repository creation, updating, 
maintenance, and use; to provide insights into text-based data repositories and 
search capabilities. 


Knowledge Management 


To provide hands-on experience with a working information and knowledge 
creation, sharing, and utilization system. 



Proceedings of the 16 ,h Annual Conference of the International Association for Information Management 



309 



334 



CONCLUSIONS AND IMPLICATIONS 

I-KSS is an application that demonstrates the potential to 
leverage Internet information in knowledge management 
and knowledge sharing processes. In organizational 
settings, it enables newly acquired strategically relevant 
information to be added to a knowledge repository and to 
automatically alert key decision makers of knowledge 
repository updates. I-KSS can play an important role in 
an organization’s environmental scanning process that is 
crucial for identifying external opportunities and threats, 
implementing strategic changes, and achieving 
organizational alignment. 

I-KSS also illustrates that commonly available 
communications applications (such as electronic mail) 
can be adapted to enhance knowledge sharing within 
organizations. I-KSS and the conceptual model on which 
it is based have the potential to facilitate theoretical and 
empirical investigations focused on the design of 
knowledge sharing systems that are capable of 
incorporating and leveraging information derived from the 
Internet. For MIS professionals, I-KSS suggests 
practical guidelines for leveraging information 
technologies to collect, analyze, store, and share Internet- 
based information. 

In academic settings, I-KSS has the potential to be used 
as a tool group-oriented information and knowledge 



acquisition, filtering, synthesis, and sharing tool. Its user- 
friendly features enable it to be effectively leveraged by 
student teams in a wide variety of IS courses to develop 
a repository of assignment-relevant information from 
Internet sources. A group member who identifies what 
he/she considers to be “new” information on Web sites, 
he/she can leverage the filtering capabilities included in I- 
KSS’s “Add” and “Update” repository management 
features to ensure that the information has not already 
been added to the knowledge repository. The group 
member can subsequently take advantage of e-mail 
capabilities to let the other members of the team know 
they should review the new assignment relevant 
information that he/she has added to the knowledge 
repository. 

As a result of using I-KSS, student team members may 
find the process needed to develop group deliverables 
easier and more satisfying/enjoyable. The 
collaborative/cooperative learning experience associated 
with developing group deliverables may also be enhanced 
through knowledge sharing tools such as I-KSS. Through 
the use of such tools, student team members gain 
firsthand insights into knowledge management processes 
and the potential utility of intranets. Figure 4 summarizes 
some of the potential impacts of knowledge sharing tools 
on student teams in IS courses. 



FIGURE 4 

POTENTIAL IMPACTS OF I-KSS AND 
OTHER KNOWLEDGE SHARING TOOLS ON STUDENT TEAMS 
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Exposure to student-centered intranets and other 
knowledge management/sharing tools in IS courses has 
been encouraged by Dun and Hackney (2000) in response 
to the increasing importance of knowledge management 
processes within today’s organizations. Demand for 
knowledge sharing tools such as I-KSS is also fueled by 
the increasing prevalence of group assignments in IS 
courses that involve Internet-based research. I-KSS 
arguably provides a more coherent and effective 
environment for the assembly of and sharing of 
information sources required by such assignments than 
many of the approaches that student teams have 
traditionally used. While other tools such as groupware, 
computer supported collaborative learning (CSCL) tools 
and computer support for distributed collaborative 
learning (CSdCL) tools approaches may be effectively 
substituted for knowledge sharing tools to facilitate 
collaborative learning and team-based information/ 
knowledge creation and sharing, I-KSS and other 
knowledge sharing tools are better positioned to help 
students master key knowledge management concepts as 
they work through the process of developing group 
deliverables. 

FUTURE RESEARCH 

I-KSS’s potential to support student teams and facilitate 
student mastery of knowledge management concepts must 
be tested. During Fall Semester 2001 student teams in a 
data communications course will be required to use I- 
KSS during the completion of a network design project. 
The network design assignment requires each student 
team to design two alternative LANs for a particular 
workspace within a specific organization and to use the 
Internet to identify the hardware, software, and 
communication media costs associated with each of the 
LANs they design. Cabling, server, hub/switch, and 
network operating system costs products, vendors, and 
prices must be identified. The assignment also requires 
the student teams to use the Internet to identify current the 
prices of backup systems and internetworking 
technologies, such as bridges, routers, and superhubs. 
Design drawings and cost information is summarized for 
each LAN is included in a paper that recommends a 
particular configuration for the organization. Pertinent 
information from Internet sources on the total cost of 
ownership, advantages and disadvantages of the 
recommended LAN configuration must also be included 
in the team project deliverable. 



Student teams will be introduced to I-KSS via a one-hour 
training session. They will also be required to complete a 
mini-exercise to introduce them to how this knowledge- 
sharing tool can be used to assemble/share the 
information needed to complete the network design 
project. At the conclusion of the project, each student will 
be asked to answer the questions included in Appendix A. 
The survey captures student reactions to I-KSS interfaces 
and student eagerness to use I-KSS to complete future 
group assignments. Most importantly, the survey prompts 
students to respond to items that assess whether I-KSS 
facilitated group information sharing and communication 
and whether the tool promotes understanding of 
knowledge management processes. 

An additional experiment is planned for Spring Semester 
2002 to assess the performance of I-KSS. Student teams 
in one section of a data communications course will be 
required to use I-KSS to complete a group network design 
project similar to that outlined above. Student teams in a 
second data communications section will be required to 
complete the same network design project without the 
assistance of I-KSS. The quantity, quality and utility of 
the information collected by groups in both sections will 
be assessed in order to determine if I-KSS adds value to 
group deliverables. Students in both sections will also be 
asked to complete a questionnaire concerning group 
processes leading up to the submission of the group 
deliverable (e.g., how easy it was to assemble and share 
assignment-relevant information; quantity and frequency 
of communication among group members, etc.) Students 
who use I-KSS will be asked to complete the student 
reaction survey in Appendix A. 

Student feedback about I-KSS’s interfaces and 
functionality will be used to improve the I-KSS 
configuration. Once configuration refinements are 
complete, the ability of this knowledge sharing tool to 
enhance the performance of teams in business 
organizations will also be tested. 
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APPENDIX A 

I-KSS REACTION SURVEY 



Select the response option that best describes your personal assessment of I-KSS. 

1 ) I-KSS helped my group assemble and share assignment-relevant information. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

2) I-KSS is easy to learn and use. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

3) Adding new information to the I-KSS knowledge repository is difficult. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

4) The e-mail component in I-KSS makes it easy to let classmates know that new information has been added to 
knowledge repository. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

5) I would like to use I-KSS for future group assignments in other courses. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

6) The user interfaces in I-KSS are easy to understand and use. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

7) I-KSS helps me understand how knowledge is captured, shared, and managed in organizations. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

8) I-KSS’s KB Search Engine makes it easy to identify important information in the knowledge repository. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

9) The Comment History, Review History, and Related Documents components are useful features of I-KSS. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

1 0) I feel that I-KSS increased assignment-relevant information sharing and communication among the members of my 
group. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

11) 1 will encourage groups in other courses to use I-KSS to collect, manage, and share information from the Internet. 

Strongly agree 1 — 2 — 3 — 4 — 5 — 6 — 7 Strongly Disagree 

12) Use the space below to specify how I-KSS should be changed/improved. 
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V 



DEMOGRAPHIC INFORMATION: 



Gender: [ ] Male 

[ ] Female 



Age: 



Education: 

[ ] Some High School 
[ ] High School Graduate 
[ ] Some College/University 
[ ] College/University Graduate 
[ ] Some Post Graduate Work 
[ ] Post Graduate Degree 



Major: 

[ ] Information Systems 
[ ] Information Technology 
[ ] Computer Science 
[ ] Computer Engineering 
[ ] Other (please specify): 



Computer Knowledge: 

[ ] No Knowledge 
[ ] Some Knowledge 
[ ] Reasonable Knowledge 
[ ] Very Knowledgeable 
[ ] Expert 



Frequency of computer use: 

[ ] Do not use computers 
[ ] Less than once per week 
[ ] Multiple times per week 
[ ] Daily, < 1 hr/day 
[ ] Daily, 1-3 hrs/day 
[ ] Daily, > 3 hrs/day 



Internet Knowledge: 

[ ] No Knowledge 
[ ] Some Knowledge 
[ ] Reasonable Knowledge 
[ ] Very Knowledgeable 
[ ] Expert 



Frequency of Internet use: 

[ ] Do not use the Internet 
[ ] Less than once per week 
[ ] Multiple times per week 
[ ] Daily, < 1 hr/day 
[ ] Daily, 1-3 hrs/day 
[ ] Daily, > 3 hrs/day 



How long have you been using computers: 

[ ] < 1 year 
[ ] 1-2 years 
[ ] 3-5 years 
[ ] 6-10 years 
[ ] 11-15 years 
[ ] > 1 5 years 



How long have you been using the Internet and WWW: 

[ ] < 1 year 
[ ] 1-2 years 
[ ] 3-5 years 
[ ] 6-10 years 
[ ] 1 1-15 years 
[ ] > 1 5 years 
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V 



Please indicate your uses of the Internet and their frequency: 





Do 


Less 


Multiple 




Multiple 




NO 


than 


Times/ 




Times/ 




T 


Weekly 


Week 


Daily 


Day 




use 










E-mail 


[1] 


PI , 


[3] 


[4] 


[5] 


Instant messaging 


[1] 


[2] 


[3] 


[4] 


[5] 


Chat-room participation 


[1] 


[2] 


[3] 


[4] 


[5] 


Internet voice communications 


[1] 


[2] 


[3] 


[4] 


[5] 


Entertainment (games, music, movies, etc.) 


[1] 


[2] 


[3] 


[4] 


[5] 


Shopping for consumer goods (without purchasing) 


[1] 


[2] 


[3] 


[4] 


[5] 


Purchasing consumer goods on-line 


[1] 


[2] 


[3] 


[4] 


[5] 


Travel Information (airline, car rental, hotel, etc.) 


[1] 


[2] 


[3] 


[4] 


[5] 


Travel Reservations/Bookings 


[1] 


[2] 


[3] 


[4] 


[5]' 


Information (news, weather, sports, etc.) 


[1] 


[2] 


[3] 


[4] 


[5] 


Investment and financial information 


[1] 


[2] 


[3] 


[4] 


[5] 


Banking on-line 


[1] 


[2] 


[3] 


[4] 


[5] 


Education 


[1] 


[2] 


[3] 


[4] 


[5] 


Research (business or academic) 


[1] 


[2] 


[3] 


[4] 


[5] 
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ABSTRACT 

Organizations are looking at ways to manage their information resources . Both the capture and the use of data 
created by the organization have come under scrutiny. Part of the concern comes from their desire to enhance the 
business process and part comes from the explosion of data available to any organization. In this context the role 
of committees is being examined both as consumers and producers of information. The present work looks at making 
corporate knowledge more useful to organizations by focusing on supporting committees. We focus on presenting 
our knowledge model and briefly look at the prototypes used to test the feasibility of our model. 



INTRODUCTION 

Knowledge is rapidly becoming a key organizational 
resource and determinant of organizational performance 
and competitiveness. Knowledge is the currency of 
wealth. Knowledge arises out of best practices, 
traditions, cultures, and other information. Knowledge 
can be classified into several forms, for example, tactile, 
syntactic, semantic, and strategic. Whatever the form of 
knowledge a firm has, unless it is managed properly, 
there can be no benefit or strategic advantage. 
Consequently, knowledge management is a process that 
organizations employ to improve performance. 
Knowledge management, however, is best described as 
a concept that explains how information is transformed 
into actionable knowledge and made available to users. 
Knowledge management enables businesses to avoid 
repeating past mistakes, to assure the reuse of best 
practices, and draw on the collective wisdom of its 
employees past and present. Knowledge management 
thus relies on gathering, organizing, refining, and 
disseminating information and knowledge. Knowledge 
is often embedded in an organization in the form of 
experiences or memory. This embedded knowledge has 
traditionally been known as organizational memory. 



Recent developments in information processing 
technologies have enhanced our abilities for building 
knowledge management systems. Knowledge is 
determined through searching, filtering, cataloging, and 
linking of information collected from a variety of 
sources and media. Information is refined in multiple 
ways and disseminated to recipients as reports, analyses, 
etc. 

A knowledge management system must systematically 
deal with the creation, integration, maintenance, and 
dissemination and use of all kinds of knowledge within 
an organization. We have developed a committee-based 
model comprising several components to manage this 
process. We see such a model as defining a portal from 
the organization to its knowledge. The strength of an 
enterprise information portal lies in its ability to organize 
disparate information in a seamless fashion. Specifically, 
the model automates identification and distribution of 
relevant content, provides content sensitivity, interacts 
intelligently with users letting them profile, filter, and 
categorize, and avails of the complex information 
infrastructure. Central to the system is the user, often a 
decision-maker or a decision making body of people (as 
in the case of a committee or group). The components of 
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the model are (1) a recorder subsystem, (2) a composer 
and builder subsystem, (3) a knowledge navigator and 
retriever subsystem, and (4) a knowledge percolator 
subsystem. The model operates on the organizational 
memory of the organization. While many vendors sell 
products meeting one or more of the features described 
above, the lack of ability to apply many of these and 
other related technologies to complex knowledge such as 
that contained in prior decisions (including the decision 
making process) and other forms of organizational 
knowledge is a problem. In this paper we present the 
model and briefly look at the prototypes used to evaluate 
the feasibility of our knowledge management model. 

In the next section we briefly look at the issues of 
developing and using organizational memory. In 
Section 3 we define and describe our committee-based 
model. Section 4 looks at the current status of our 
prototypes for testing the feasibility of the model. 
Section 5 concludes the paper. 

ORGANIZATIONAL MEMORY 

Organizational memory has been described to refer to 
corporate knowledge that represents prior experiences 
and is saved and shared by users. It may be used to 
support decision making in multiple task and multiple 
user environments. The concept encompasses technical, 
functional, and social aspects of the work, the worker, 
and the workplace [Durstewitz 1994]. Organizational 
memory includes stored records (e.g., corporate manuals, 
databases, filing systems, etc.) [Ackerman 1996] and 
tacit knowledge (e.g., experience, intuition, beliefs) 
[Nonaka andTakeuchi 1995]. Walsh and Ungson [Walsh 
and Ungson 1991] refer to organizational memory as 
stored information from an organization’s history that 
can be brought to bear on present decisions. By their 
definition, organizational memory provides information 
that reduces transaction costs, contributes to effective 
and efficient decision-making, and is a basis for power 
within organizations. Researchers and practitioners 
recognize organizational memory as an important factor 
in the success of an organization’s operations and its 
responsiveness to the changes and challenges of its 
environment (e.g., [Angus et. al 1998; Bright et. al 1992; 
Huber 1991; Huber et. al 1998; Stein 1995; Stein and 
Zwass 1995]). 

Information technologies (IT) contribute to the 
possibility of automated organizational knowledge 
management systems in two ways, either by making 
recorded knowledge retrievable or by making individuals 
with knowledge accessible [Ackerman 1996]. An 



organization’s knowledge, explicitly dispersed through 
a variety of retention facilities (e.g., network servers, 
distributed databases, intranets, etc.) can make the 
knowledge more accessible to its members. Stein and 
Zwass [1995] suggest that an extensive record of 
processes (“through what sequence of events?”), 
rationale (“why?”), context (“under what 
circumstances?”), and outcomes (“how well did it 
work?”) can be maintained with the help of IT. The 
availability of advanced information technologies 
increases the communicating and decision making 
options for potential users. 

An organizational memory supported by information 
technology provides several advantages The contents 
that are stored in information systems are explicit, can be 
modified promptly, shared as necessary, and changes can 
be propagated quickly and easily. Information systems 
should be however be designed to augment the 
interaction between knowledge seekers and information 
providers as it would lead to higher levels of 
organizational effectiveness and learning [Huber et. al 
1998]. Technological changes and shifting demands 
make rapid learning essential in organizations. The 
advent and increasingly wide utilization of wide area 
network tools such as the Internet and World Wide Web 
provide access to greater and richer sources of 
information. Local area networks and intranets give 
organizations a way to store and access organization 
specific memory and knowledge. Used effectively, these 
tools, support the notion of organizational memory. 

Sandoe et. al [1998] use Giddens’ definition [1984] of 
organizational memory to distinguish among three types 
of memory, namely discursive, practical and reflexive. 
Discursive memory is a collection of stories, anecdotes, 
reports, and other public accounts. Practical memory is 
the unarticulated know-how — the skills and practices 
that comprise the bulk of routine interactivity. Memory 
that is automatically invoked in organizations without 
any requirements for conscious thought or discussion 
such as organizational norms is considered reflexive. 
Sandoe et al. [1998] argues that there is a continuum of 
characteristics that stretches between discursive memory 
on one end and reflexive memory on the other. 
Discursive memories are oriented to be flexible while 
reflexive memories are focused on yielding efficient 
memory use. Organizations have to balance the need for 
flexibility against the need for efficiency. 

Sandoe et al. [1998] treat IT based organizational 
memory as discursive. They argue that although IT- 
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based memory operates at a discursive level, IT makes 
the discursive process of remembering more efficient by 
reducing the costs and effort associated with storage of 
and access to an organization’s memory. IT changes the 
balancing point in the trade-off between efficiency and 
flexibility, permitting organizations to be relatively more 
efficient for a given level of flexibility. Another 
advantage of IT-based memory is the opportunity to 
provide a historical narrative (or rationale) for significant 
organizational events that would otherwise be 
remembered in non-discursive form. Reliance on IT- 
based memory, on the other hand, allows an organization 
to act in a rational manner through the discursive access 
to its major historical events and transformations. 

Organizational Memory Systems (OMS) 

Mandiwalla et al. [1998] define an organizational 
memory system (OMS) to include a DBMS, a database 
that can represent more than transactional data, and an 
application than runs on top of the DBMS. They further 
describe the generic requirements of an OMS to include 
different types of memory, including how to represent, 
capture, and use organizational memory. The data shown 
below lists these requirements. 

OMS Design Issues 

Designing the ideal OMS is a difficult task, especially as 
definitions, technologies, and usage contexts continue to 
shift and evolve. Mandiwalla et al. [1998] identified 
some of the barriers and issues facing designers. 

• Focus: Reconciling group, inter, and intra 

organizational perspectives of OM. 



• Quantity: Balancing comprehensiveness with storage 
constraints. Incorporating video data quickly tilts the 
balance away from comprehensiveness. Increasing 
comprehensiveness also increases the potential for 
information overload. 

• Filters: If you cannot store everything then who 
decides what to store? What is the mechanism and 
criteria? How do you avoid bias? 

• Role of individual memories: Where do individually 
held memories fit in? Are they redundant? How can 
they be used? What are the legal and social 
implications? 

• Storage: OM typically implies some type of storage. 
Information storage will in the foreseeable future 
always involve costs such as the actual storage 
medium, the time needed to access the selected 
medium, and the administrative cost of maintaining 
the information. Organizations will need tools that 
that will help them evaluate the cost benefit of storing 
memory. For example, 1 second of video (30 frames) 
needs about 27K of space. This means that about 3 
hours of video could be stored on a 10-Gigabyte 
medium. It is unlikely that storage costs will decrease 
sufficiently in the near term for this to be a non-issue. 

• Retrieval: Widely held assumptions about data imply 
that the more OM we store the harder it becomes to 
locate a specific memory item of interest. Therefore, 
OM conceptual models will need a built in filtering 
mechanism. 



Types of memory 


Meta data 

Structured data (records) 
Semistructured data (documents) 
Unstructured data (audio/video) 
Temporal data 


Types of representation 


Structure and internal organization of data 
the representation of data in the user interface 


Capture schemes 


Transparent versus assisted 
Hidden versus open 


Use 


Time and Space considerations 
Activity driven 
Coordination needs 
Boundaries including security 
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• Integration/Re-integration: If information about the 
same topic is stored in multiple formats — say in 
database and multimedia format, users will need to 
tools to re-integrate or “re-understand” and 
synchronize the memory. 

In the next section we look at a knowledge management 
system that uses the concepts from organizational 
memory and applies them to the committee level. 

COMMITTEE-BASED KNOWLEDGE 
MANAGEMENT MODEL 

A knowledge management system must systematically 
deal with the creation, integration, maintenance, and 
dissemination and use of all kinds of knowledge within 
an organization. We have developed a committee-based 
knowledge management model comprised of several 
components to manage this process. The strength of our 
committee-based model lies in its ability to organize 
disparate information in a seamless fashion. Specifically, 
the model automates identification and distribution of 
relevant content, provides content sensitivity, interacts 
intelligently with users letting them profile, filter, and 
categorize the complex information infrastructure. 
Central to the system is the user, often a decision-maker 
or a decision making body of people (as in the case of a 
committee or group). The components of the committee- 
based model are (1) a recorder subsystem, (2) a 
composer and builder subsystem, (3) a knowledge 
navigator and retriever subsystem, and (4) a knowledge 
percolator subsystem. A block diagram of the 
relationship between organizational memory and the 
components of the model is shown in Figure 1 . The data 
comprising the organizational memory provides the basis 
of any knowledge extraction. In the remainder of this 
section we examine the main components of the model. 

Recorder 

Organizational processes such as meetings, sequences of 
tasks, operating procedures and policies, etc., are 
potential sources of knowledge. Recording or capturing 
a knowledge production process is the first step towards 
making knowledge available for future use. Capturing 
the decision making process, as in the case of capturing 
a focus group meeting, allows for further analysis of the 
process. Similarly, recordings of events in a process 
environment (e.g., software development or system 
maintenance) can be used to maintain examples of best 
practices. 



FIGURE 1 

RELATIONSHIP BETWEEN COMPONENTS 
AND ORGANIZATIONAL MEMORY 




A recorder is a tool for capturing the knowledge genera- 
tion process. Since our model is committee based, we see 
the recorder as capturing committee meetings. There are 
numerous formats for the recorder. As support software 
improves (e.g., dictation software), the range of 
possibilities will continue to expand. In today’s 
environment the possibilities depend on the media used 
to conduct the meeting. If the media is the Internet or 
conferencing software, then a record of the meeting can 
be obtained by storing the electronic form of the 
transmissions. For traditional meetings, it is possible to 
record the entire committee meeting using meeting 
capture software [Miller and Nilakanta 1997]. You can 
capture the complete meeting script, including sound, 
images, and documents that are part of the meeting 
(decision making) context. Once the recording is 
complete, the data is prepared for playback. Preparation 
includes eliminating errors, replacing missing data, 
synchronizing the sound with the flow of events, and 
adding security tags. A recorder is a valuable 
complement to any collaborative tasks. 

Composer and Builder 

Organizational knowledge ranges from structured data 
(as provided by on-line transaction processing systems- 
OLTP) to unstructured data such as user comments, 
email messages, reports, images, etc. The majority of the 
structured knowledge in an organization resides in its 
data warehouses and production databases. Proponents 
of decision support technologies have focused on not 
only retrieving information from vast databases, but also 
in identifying relationships among the different 
information sets. Only recently has this vision started to 
take shape. Data warehouses and data mining are fast 
becoming the knowledge sirens of the nineties. 
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Designing and building a data warehouse has been an 
expensive and error prone process. It does not have to be 
that way. Today, the designer must have intimate 
knowledge of the database schema to generate a valid 
data warehouse. With rapid turnover of technical 
professionals, organizations have difficulty in satisfying 
this goal today. While more complete meta-data 
(information about the data) could help, a lack of 
industry standards is also a major hurdle. The fact that 
commercial warehouses are value based (record- 
oriented) and much of the most valuable organizational 
data is unstructured also limits the value of today’s 
warehouses in the knowledge management process. 
Object-oriented data warehouses [Miller et. al 1997, 
1998] will help bridge this gap, but much work still 
needs to be done. 

Another large amount of data (especially things like 
reports, email and web pages) may not belong in the data 
warehouse environment. Supporting these data types 
requires additional design criteria beyond the 
development of a module to create data warehouse 
designs. 

Warehouse composer. Research in the database domain 
has developed several models and algorithms that are 
useful for designing data warehouses with little 
intervention of a data warehouse expert required. The 
warehouse composer uses meta-data to guide the 
designer to construct any desired warehouse table. The 
data warehouse composer is responsible for advising the 
user if a feasible warehouse solution is possible, warning 
if none exists. The issue is to test the queries required to 
generate the warehouse table for the lossless join 
property. Failure to test for the lossless join property 
could result in creating warehouse tables with erroneous 
data relationships. Warehouse sources can be single 
databases or multiple, heterogeneous data sources 
(relational DBMS like Oracle, SQL Server, flat file 
systems like VS AM, or any other type of legacy system). 
When the data sources are many and varied, the 
warehouse modeler must be able to integrate the 
semantics of the data sources and bridge the distributed 
nature of the environment. 

Warehouse builder. The warehouse builder is a sub- 
component that automatically populates the composed 
data warehouse design. It must be able to automatically 
populate the warehouse with data extracted from 
multiple, distributed data sources once the warehouse 
composer has generated the design of the warehouse 
table. 



Additional data sources. While the recorder is the 
source of much of the company’s internal data, other 
data, such as email and additional reports, also need to 
be captured. External data (e.g., reports, web pages) are 
also part of any company’s knowledge base and must be 
either collected or made accessible by users. 

Knowledge Navigator and Retriever 

Two subsystems are needed in our model to provide 
access to both forms of data. First, we need a knowledge 
navigator to allow users to navigate through the recorded 
committee meetings. Second, a retriever subsystem is 
necessary to retrieve information from the organizational 
memory that does not require navigating a committee 
meeting. 

Knowledge navigator. Navigation through a 
corporation’s knowledge base is a critical component of 
any knowledge management environment. In our 
committee-based approach it is necessary that the users 
be able to look at all or part of any meeting (depending 
on the security clearance of the user). The committee 
knowledge navigator is a playback subsystem allowing 
for selective replay of the meeting or other knowledge 
creation processes. Selections can be made on the basis 
of keywords, participants, or other parameters of choice. 
Unlike other screen-capture movies, the knowledge 
navigator must give the user increased flexibility in 
choosing portions of the process to view or learn. 
Moreover, it must be possible for key parts of the 
process to be isolated (based on rankings, keywords or 
security levels) and studied. 

Knowledge retriever. As an organization builds its 
knowledge base using both structured and unstructured 
data, on line knowledge management systems capable of 
providing this knowledge to users is needed. Typically, 
some form of universal query environment that can work 
with any type of data is needed. These systems can be 
built around proven technologies such as relational and 
object database management systems, data warehouses, 
distributed objects (CORBA), and intelligent agents. 

Retrieving information from standard data warehouses 
or transaction systems is a trivial process. But, when the 
information is complex (email messages, documents, 
etc.) and it is widely distributed (internal and external to 
the organization), simple retrieval schemes will generate 
millions of pieces of information. Not only is the sheer 
volume of information retrieved a problem, the 
relevancy of the retrieved information is also 
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questionable. Unless the retrieval tool can filter out 
unwanted and irrelevant information and suggest a 
relevancy index the tool will be of little use. 

To support the data that are not directly tied to an 
individual committee meeting, it is necessary to provide 
one or more interfaces that give the user access. Today 
it is very likely that multiple interfaces would be used 
e.g., an interface that gives access to an existing data 
warehouse or production database and an interface that 
gives the user access to textual data. In the future as 
multidatabase systems [Bright et. al 1992] mature a 
single interface to several heterogeneous data sources is 
likely to reduce the need for multiple user interfaces. 

Knowledge Percolator 

Benefits accrue to organizations when their members can 
learn from prior experiences. Learning from best 
practices and experiences requires that the user has easy 
access to relevant information, can ascertain causes and 
effects, and derive generalizations. Building a web of 
antecedents and consequences creates the collective 
memory of organization. 

The knowledge percolator is a software suite that enables 
the percolation of “wisdom” from prior experiences. 
Several learning algorithms (e.g., genetic algorithms, 
pattern recognition, etc.) would be used depending on 
the task’s context to glean the underlying decision 
process or workflow. 

IMPLEMENTATION OF THE 
COMMITTEE-BASED MODEL 

To test the feasibility of the model presented in the 
previous section, we implemented prototypes of the 
components of the model. A block diagram of a typical 
problem domain is shown in Figure 2. The remainder of 
this section briefly looks at some of the key features of 
our prototypes. 

Recorder 

To capture meetings, we have implemented acknowledge 
recorder designed to capture individual committee 
meetings [Miller and Nilakanta 1997]. The capture 
phase starts at the meeting. The sounds of the meeting 
are captured in a .wav file. In addition to the captured 
sound, the recorder generates a command file and 
captures any on-line images that have been presented at 



the meeting. A block diagram of the process is shown in 
Figure 3. 



FIGURE 2 

BLOCK DIAGRAM OF A 
TYPICAL PROBLEM DOMAIN 




FIGURE 3 

CAPTURE PHASE OF THE RECORDER 




Meeting sound* 



f E E l 




image Collection 



The capture tool is a Visual Basic program designed to 
make the task of capturing the semantics of the meeting 
a reasonable task. The tool is event driven and is based 
on the following set of fixed events. 

Sound capture. Here the notion is that the recorder will 
need to capture events such as changes in speakers 
and/or changes in topic. The tool makes it easy for the 
recorder to capture information about who is presenting, 
relevant keywords, meeting time, and a relevancy index 
(rank). Initiation of this event assigns the current time 
stamp to the command file entry. 

On-line image capture. The recorder can capture an on- 
line image by initiating this event. The image is 
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captured from the screen and placed in the image 
collection. In other words the image is placed in a file 
and the file name along with a system generated image 
id are placed in a file called the image file. The program 
also creates an entry for the command file consisting of 
the current time stamp and the image id. The recorder 
can supplement this information with the presenter of the 
image (who), meeting time (when), relevant keywords 
(what), and a relevancy index (rank). 

Hardcopy capture. This event allows the recorder to 
capture the relevant information about any 
transparencies or charts used during the meeting. The 
information captured is similar to the on-line image 
capture with the obvious exception that the recorder will 
need to convert the hard copy images to an on-line 
format, along with the information on who, when, what, 
and why. 

Document capture. This event is similar to the 
hardcopy image capture event except for the fact that the 
document will be viewable as a unit and the only entry 
in the command file will be made when the document is 
handed out. 

Beyond the on-line image capture event (which both 
captures an image and creates an entry in the command 
file), the main function of the last two events is to create 
an entry in the command file. As each event is recorded, 
a timestamp is added. Additional comments, keywords 
and a rank, indicating relevancy of the content to the 
goal of the meeting are also added by the secretary. Each 
event has a start and end point. A sample screen is 
shown in Figure 4. 

FIGURE 4 

CURRENT CAPTURE TOOL SCREEN 




At the end of the meeting, the meeting secretary (we 
used the term recorder for the meeting secretary in 
[Miller and Nilakanta 1997] and have maintained it here 
for consistency) converts the documents, transparencies 
and charts into an electronic format. The current 
approach used in our design is to scan anything that the 
participants can’t provide in electronic form. Individual 
images and documents are added to the image and 
document collections, respectively. Note that the format 
of both collections is a file of pointers to the individually 
stored objects. The result is that at the end of the capture 
phase, a command file, an image collection and a 
document collection are ready for additional processing. 

We have implemented a post capture tool to allow the 
secretary of the meeting to incorporate more relevancy 
information into the command file and to synchronize 
the command file with the sound file. The first aspect of 
the post capture phase is that the recorder uses a post 
capture tool to replay the meeting based on the captured 
semantics. During this phase, the recorder can correct 
obvious command entry errors and insert the post 
meeting rank values for the objects. Note that the 
correctable part of a command is the time stamp, the 
assigned participant(s), and the relevant keywords. The 
relevancy index (rank) assigned by the recorder during 
the meeting can not be changed. The security level of 
the command file entries are added during the post 
capture phase. A block diagram of this phase is shown 
in Figure 5. 



FIGURE 5 

ORGANIZATION OF POST CAPTURE TOOL IN 
CURRENT IMPLEMENTATION 




The result is that a meeting script is created for each 
committee meeting. The meeting script can then be used 
by the knowledge navigator subsystem to allow users to 
revisit meetings through selected playback. 
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343 BEST COPY AVAILABLE 




Knowledge Navigator 

A prototype of a meeting viewer has been developed to 
take the data captured by the recorder subsystem and 
present the meeting via a web interface. The software 
transforms the command file produced by the recorder 
into a new command file based on the user’s security 
clearance. All entries that violate his/her security 
clearance are removed. Once the new command file is 
created, the user can either view the restricted portion of 
the meeting in its entirety or search it for keywords and 
view the portion of it that is close to the location of the 
keyword in the command file (the meaning of close is 
specified by the user in minutes). The prototype plays 
the sound of the meeting, places and removes images 
from the screen depending on their start/stop time, and 
makes documents available to the user at the point in the 
meeting that they were handed out. The user can replay 
portions of the meeting script. 

Composer and Builder 

Two prototypes have been designed and implemented to 
this point for designing and building data warehouse 
tables. The first prototype provides a click and point 
interface that the user can use to choose the format of a 
warehouse table from the meta-data for a set of source 
relational databases. Once the user has chosen the 
desired data warehouse attributes, the prototype uses a 
query generator that generates a lossless query over the 
relational databases or returns a message that no lossless 
query is possible. If the data warehouse attributes span 
more than one database, the query can be partitioned to 
create a set of queries that can then be used to obtain the 
required data to populate the data warehouse table. The 
lossless test has been designed to make use of the table 
names, attribute names and functional dependencies (if 
they are available) from the production database meta- 
data. 

A second prototype has been developed to build the 
warehouse table from the query generated by the 
previous prototype. In this new prototype once a query 
is generated, the query is partitioned and a set of mobile 
software agents are generated to go to the data source 
locations, generate the required data, and return it to the 
interface for storage in the data warehouse. The data is 
automatically added to the new warehouse table when 
the agent returns to the prototype’s interface. To this 
point, the prototype supports both relational databases 
and batch based legacy systems. 



Knowledge Retriever 

Our prototype for the knowledge retriever fetches 
information using an intelligent search process. 
Knowledge retriever is based on well-grounded research 
in the area of information retrieval. Our prototypes use 
the vector space model. In addition to comparing 
queries and documents, we make use of user or 
committee profiles in our prototypes. A profile is a list of 
possibly weighted terms provided by a user, which 
reflects the users or committee’s long-term interests. 
Initially, a profile can be created manually or based on a 
set of relevant documents. Documents, queries and 
profiles all are represented by term vectors at retrieval 
time. The full system consists of three parts: indexing 
the documents, weight assignment and retrieval or 
filtering. 

During the retrieval phase each query, document and 
committee profile is represented by a list of weighted 
terms to reflect the relative importance of terms. The 
weight of term tj in document di is defined by wtfij) = 
tf(ij) * log(N/df(j)), where N is the total number of 
documents in a collection and tf(ij) and df(j) are term 
and document frequencies as defined above. 

The similarity between a document and a query or a 
profile is represented by the mathematical properties of 
the vectors. For a document D and a user query or a 
profile Q, let DD = char(D), QQ = char(Q), then the 
similarity between d and q is defined by 

DD • QQ 

SIM(D, Q) = cos (A ngleOf(DD, QQ)) = 

//DD// * //QQ// 

where 

DD QQ= ZDDfiJ * QQ[iJ; 

//DD// = sqrt( ZDD[i] * DDfiJ ) and 

//QQ// = sqrt( L QQ[i] * QQfi] ) are the norms of 

vectors DD and QQ, respectively. 

All the documents are ranked by this similarity 
coefficient. Documents whose similarity coefficients 
exceed a predetermined threshold are retrieved. 

A relevance feedback mechanism has been provided to 
modify the original query or profile to achieve better 
retrieval performance. Based on the feedback from 
users, term weights in the committee profile are changed 
to reflect the committee’s actual interests. 
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The current prototype has been implemented using 
mobile software agents to connect the user interface with 
the document collections. An earlier version of the 
prototype [Hu et. al 1998] used a more typical interface 
to a file of documents to support the knowledge 
management process. 

Knowledge Percolator 

There are numerous opportunities in an organization to 
generate knowledge. Since our approach is committee- 
based, we have focused to this point on analyzing 
committee decisions. Our current prototype is only the 
first step in developing this aspect of the model. The 
obvious choice to start analyzing the committee decision 
process is to look at the information in the data generated 
by the committee meeting. 

In our current prototype we made use of a neural 
network to model the decision process of committees. 
The command file generated by the recorder contains the 
necessary information, but can not be used directly. It 
requires transformation into a format that can be used by 
the neural network. To aid in the transformation 
process, the prototype incorporates a set of 
transformations based on sliding a window over the 
records in the command file. 

The command file entries are used by the transformation 
to generate the specified statistics for each member of 
the committee. The prototype assumes that the 
committee membership consists of the union of the 
attendees of the set of meetings that are being processed. 
The decision-variable is set to true if a decision has 
been reached in the command file records covered by the 
window and is set to false if no decision has been 
reached. 

The focus of the current software is to look for activity 
patterns in the meeting that result in decisions. The 
transformed file is used to construct a neural network. 
The decision-variable serves as the classifier in the 
training set for the neural network construction. Once the 
prototype has build the neural network, the neural 
network can be used to generate a set of rules for the 
decision making process within the committee. The 
motivation behind this approach is to develop a 
methodology for setting up committee meetings to 
optimize the decision process. 



CONCLUSIONS 

A model of a knowledge management environment has 
been given. The model takes a committee-based 
approach and uses committee meetings as the primary 
data collection point. The assumption is that more 
traditional forms of data (databases, data warehouses, 
and report libraries) are easy to generate and the major 
concern is to incorporate them in with the knowledge 
management process. Existing prototypes for the 
components of the model have been briefly overviewed 
to address the feasibility of the model. 
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ABSTRACT 

The use of advanced learning technologies enables various new forms of collaborative learning and group 
assessment. This paper outlines a radically different model of student assessment to that traditionally used in most 
universities and other educational institutions. The model is particularly suited to those using , or seeking to use , 
collaborative learning techniques, which are themselves in turn greatly enabled by the use of asynchronous learning 
technologies. The advantages and disadvantages of the assessment model are discussed, and various methods of 
implementing the model are suggested. 



INTRODUCTION 

Suppose that the instructor has a choice of two possible 
models of course delivery, and can reasonab ly predict in 
advance that if the first is employed, then 80% of 
students will pass, with varying grades (A, B, C, D etc), 
and that 20% of students will fail; and that if the second 
is employed, all 100% will pass, but mostly with low 
pass grades (Cs and Ds). Suppose furtherthat the grades 
are actually a true and accurate reflection of the students’ 
learning. Which model should be preferred, and why? 

This is not a question that has been m uch debated in the 
literature, and the correct answer is not obvious. For 
example, suppose the former is preferred. Would the 
decision be different if the latter were to produce mainly 
Bs and Cs instead of Cs and Ds? What if the former 
produced only 70% passes? 60%? And so on. 

The mean or median grade could be used to prefer one 
method over the other. If these are roughly the same in 
the two models, however, thatcannot assistthe decision. 
In such cases the fonuer method, with a variety ofgrades 
is often preferred perhaps because that is less likely to 
be questioned at subsequent examiners meetings! 



Atsome point, however, a decision m ay have to be made 
between encouraging a variety of grades (usually 
considered a good thing) and increasing the number of 
students achieving passing grades (also usually 
considered a good thing). 

THE CAPITALIST MODEL OF ASSESSMENT 

There is no such thing as a standard method of 
assessment common across university courses . Rather, 
methods of assessment vary enormously from course to 
course, subject to subject, and university to university. 
For example, assessment may be comprised entirely of 
an end-of-term examination, or may be comprised of a 
number ofassignmentsplus anend-of-term examination, 
or a number of assignments only. Assignments can be 
theoretical or practical in nature. The end-of-term 
examination may be open book, or closed book, or of a 
take-home type. All assignment items may count 
towards the final grade, or only a selection. And so on 
through many other possible variations. 

Nevertheless in all of these cases the final grade is 
usually based on the student’s own work, as 
demonstrated in one form or another throughout the 
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course. Students often “compete for grades even when 
assessment is criterion-based, and no element of 
competition actually exists. 

In what follows, this will be referred to as the capitalist 
model of assessment. 

COLLABORATIVE LEARNING 
AND THE CAPITALIST MODEL 

Collaborative learning is hardly a new topic. The 
importance and relevance of social interaction to an 
effective learning process has been stressed by many 
theorists from Vygotsky [18], through advocates of 
situated learning such as Lave and Wenger [10], and has 
been confirmed by many more recent researchers and 
practitioners, such as Kagan [7] and Johnson and 
Johnson [6]. 

An up-to-date review of the research and the long history 
of peer/collaborative learning can be found in [1 1], A 
small but select annotatedbibliography on collaborative 
learning can be found at [12], while an excellent list of 
strategies and tips for those interested in introducing 
collaborative learning can be found at [2]. 

Salomon [17] amongst others has pointed out that 
despite the mass of literature praising collaborative 
learning, teams very frequently do not work well, and 
lists as common problems the “free rider effect [9], the 
“sucker effect [8], the “status sensitivity effect [3], and 
the “ganging up on the task phenomenon [1 6]. 

The use ofcollaborative learning techniques specifically 
within certain topic areas has not received much 
attention, though definite benefits have been found, for 
example, with their use in the teaching of a computer 
science curriculum [21]. 

This paper exte nds the topic of collaborative learning, to 
include, as an integral component, group assessment. 
Webb [19] lists six positive reasons for employingsome 
form of group assessment. It is suggested here that there 
may also be othermore important reasons specifically, 
that its use can have asignificantly positive effect on the 
learning outcomes of the students. 

FROM COLLABORATIVE LEARNING 
TO GROUP ASSESSMENT 

It is important at the outset to distinguish collaborative 
learning from group assessm ent. There are a number of 



forms of collaborative learning that can be used, from 
small informal group discussions where the groups vary 
from week to week, through to formal group projects 
where the groups remain fixed throughout the term. 

Even in such cases, assessment may still be carried out 
on a completely individual basis. 

Where this is not the case, and some form of group 
assessment has been used, many schemes have been 
devised to try to award appropriate grades to particular 
individuals for example, by dividing the group work 
into separate items for which a particular individual is 
responsible, or by having the group members report on 
who provided the most input within the group, or a 
variety of other factors. 

One particularly noteworthy case of implementing 
collaborative learning in an asynchronous learning 
environment is the so-called “radical model [1 3,14,15]. 
The radical model dispenses with traditionalface-to-face 
teaching almost entirely, and places the emphasis on the 
students themselves to learn within agroup setting, using 
the Web for resource material and email discussion 
groups for communication and presentation of 
assessmentitems, with the instructorproviding guidance 
and feedback as required. 

At the beginning of the course, the instructor randomly 
assigns students into groups. Each group isassigned one 
of the weekly topics, and has to make a single online 
presentation. Students are assessed not just for their 
group’s presentation but also for their comments about 
other presentations. Each group presentation is also 
assessed on the quality of the discussion that follows. 
Typically, by the end of semester, students will have 
received over 100 inputs on their work from other 
students in the group, other groups, and the instructor. 

In the last week ofterm, stude nts are invited to submit a 
recommendation in writing on each other’s group 
performance. The instructor considers any such 
recommendations when allocating individual marks for 
group performance to members of the group. A student 
that a group decides did not contribute sufficiently may 
as a result suffer a reduction in mark. 

Differentassessment criteria may beused for example, 
for the electronic presentation, clarity and structure of 
presentation, originality of ideas, and ability to 
substantiate arguments by relevant data; for other 
contributions, understanding the arguments that are 
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made by other presenters, linking them to the relevant 
literature, and makingpertinent critical comments about 
these arguments. 

The students’ final marks are based on a combination of 
their group work throughout the semester, and their 
performance in an end-of-semester examination. 

In common with some other forms of collaborative 
learning, the radical model points the way towards other 
possible forms of assessment in an asynchronous 
learning environment. However, as presently constituted, 
the model still represents an example of the capitalist 
model of assessment, since the grade awarded is based 
on the standard paradigm of attempting to assess the 
individual’s own efforts, even within the context of an 
online collaborative learning environment. 

THE SOCIALIST MODEL DESCRIBED 

In the socialist model of assessment, achieving the best 
outcome for a particular individual or small group of 
individuals is regarded as secondary to achieving the 
best outcome for the class as a whole. 

In all instantiations of the socialist model, effort is 
expended in ensuring that the most able students assist 
the least able. In this way, knowledge and skills can be 
successfully passed on outside the presence real or 
virtual of the instructor. 

This is often achieved via various forms of collaborative 
learning, where it is hardly controversial. To incorporate 
the samephilosophy into group assessment is, however, 
another matter. 

At its minimalist, this philosophy can mean dividingthe 
class into groups, and assessing each group as a whole, 
without attempting to differentiate the grades of the 
individuals within the group. Providing this strategy is 
made clear at the start of the process, it is clearly in the 
best interests of the individuals within the group to 
ensure that the group as a whole performs to the best of 
its ability. Unfortunately, this is often achieved in 
practiceby one or two members of the group performing 
the bulk of the work, and in such cases it is common for 
the weakest members of the group to learn little or 
nothing [8,9]. 

Other strategies can be employed to overcome this 
difficulty, however. The first is to make it known that 



one or more member(s) of the group will be chosen at 
random to represent that group in each item of 
assessment, be that a written test of some sort, or an oral 
examination, or whatever. Another strategy is to let it be 
known that the instructor will deliberately select the 
member(s) of the group perceived to be weakest in that 
subject area on each occasion. In either event, all 
members of the group are awarded the m arks attributable 
to the selected member(s). 

Hence there is enormous incentive for skills to be passed 
amongst the group. To avoid any “rigging of the 
system, students who withdraw prior to an assignment 
task should not be eligible for any mark for that 
assignment, while all other members should receive 
equal marks. 

What if one or more members of a group are particularly 
weak? This unfortunate, but not uncommon, situation 
can be remedied in part by ensuring that the membership 
of each group is changed for each assessment item. 

Another factor that can be varied is the allocation of 
students into groups. For example, whether to (a) let 
groups decide membership for themselves, or (b) 
randomly allocate students to groups, or (c) allocate 
students to groups according to some prearranged 
strategy. The composition of each group plays a far 
greaterrole within socialistmodels of assessment, andso 
this decision assumes some importance. 

In (a), the socialist model is mitigated to some extent, 
since there is some evidence that the most able students 
will seek to group themselves together by choice, 
leavingthe lesser able to form their own groups. In such 
cases, the final range of grades is likely to be not 
dissimilar from a standard capitalist model. 

In (b), the random option, the socialist model is likely to 
work best since both the perception and the reality is 
that one’s fellow group members are there as a result of 
the luck of the draw. It is therefore incumbent on each 
group to work together as best as it can. 

The danger with (c) is that, whatever strategy is 
employed, the final group structure can be perceived as 
having been deliberately set up to advantage or 
disadvantage certain students. It is therefore a high-risk 
strategy for the instructor, even if the method of 
selection is made transparent. 
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ADVANTAGES OF THE SOCIALIST MODEL 

The socialist model has so far been suggested primarily 
as a means of assisting the lesser-able students to achieve 
better grades than might otherwise be expected; and it 
has been implied that this usually occurs at the expense 
of exceptional grades for those more able. 

However, it is quite possible that the socialist model of 
group assessment might in fact advantage all students. 
It is often said by academics and instructors generally 
that the best way to learn a subject is to teach it, and for 
good reason when teaching, one needs to gain a 
thorough knowledge of the subject, not only to prepare 
material, but also to be able to answer questions 
confidently. No matter how good the preparation on the 
part of the instructor, further questions will almost 
inevitably arise during the course of instruction, thus 
leading to an even better understanding. 

It seems nai'vein the extremeto assume a similar process 
will not occur when the students themselves take on the 
role of instructors to other students within their group. 
Thus, it can be expected that in many cases the socialist 
model of assessment will increase the learning of all 
students within the group, and not just those who are 
least able to learn for themselves. This conclusion seems 
to be supported by previous research in this area. 

For example, Webb and Sugrue [20] report that 
“amongst groups with above-average students... .the 
higher level ofdiscussion translated into an advantage in 
the achievement tests for the below-average students (in 
those groups) , both when they were tested on a group 
basis and also individually. On the other hand, “high 
ability students performed equally well in heterogeneous 
groups, homogeneous groups, and when they worked 
alone. Both of these results have also been shown in 
different contexts by other researchers [1,4,5]. 

THE MIXED MODEL 

Despite the possible advantages listed so far, it is 
unlikely that many instructors will be prepared to 
embrace the socialist model whole-heartedly, for fear of 
falling foul of either aggrieved students or institutional 
procedures, or both. 

There is a half-way house, however: in a mixed model 
of assessment, the pure socialist m odel is combined with 
the capitalist model. For example, one valid mixed 
model would be for each student's final grade to be 



dependent upon a number of group assessments 
throughout the semester, and an indi vidualexamination. 
Clearly in such cases the weighting of each can be easily 
varied from zero to one hundred per cent, thus allowing 
the instructor to select the appropriate mix with which 
they feel most comfortable. 

SUMMARY 

A non-standard model of student assessment has been 
described, which is best suited to those courses where 
collaborative learning is already in use or is being 
seriously considered. 

It is of course of vital importance that the students be 
made fully aware from thecommencement ofthe subject 
as to the assessment model to be employed, and 
understand its implications for the class as a whole and 
for themselves as individuals. The use of a form of the 
socialist model described here can then be expected to 
benefit the less-able students, resulting in an increase in 
the percentage of students able to achieve passing 
grades . 

Since a shift to the socialist paradigm may seem too 
radical a step for many instructors, a mixed model has 
been described which may enable experimentation, and 
hence the gathering of empirical results in this area. 
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ABSTRACT 

Despite the increas ed emph asis on team work in the academic environment, managing a student team so that the team 
process is effective remains problematic . In fact, some professors believe we are teaching students ineffective team 
behavior such as “free loading” or relying on star per formers and procr as tination . Most research on student team 
process uses students as surrogates for business world teams and draws conclusions for that domain. This study 
examines prior research on student teams, performs a meta-analysis of this literature, derives a set of interventions 
found to be effective with student teams, and proposes a planned application of these interventions. 



INTRODUCTION 

According to Jones (1996, 80), “inappropriate use of 
teams can undermine the educational process so badly 
that learning does nottake place, students learn how not 
to learn, students build an attitude of contempt for the 
learning process.” Indeed, satisfaction with a team- 
based MBA program was negatively associated with the 
cognitive abilities as measured by GMAT scores of 
students (Baldwin et al, 1997, 1382, 1384,1385): 

It seems likely that the most talented students 
may feel burdened with responsibility for 
helpingothers and feel they are minimizing their 
own learning or development , , , Conversely, 
the students performing at the lowest level enjoy 
a more naturally occurring level of support in a 
team-based structure and may consequently 
report higher levels of satisfaction with their 
experience . , , 

The negative association of workload sharing 
with team grades suggests that in a significant 
number of teams, a few stars carried 
dispropo rtionate shares of the teams’ workloads. 

This finding would also help account for the 



lower satisfaction reportedby students with high 
cognitive ability. It may also confirm the fear of 
the M.B. A. instructors that say a bright student 
skilled in finance might end up doing most of 
the financial analysis for team assignments at 
the expense of the learning of others. 

Jones (1996, 85) concurs and finds that “the high 
performing members of the [student] team get a 
workout,” He cautions against “confusing business and 
educationenvironments.” Many of the business controls 
for performance are lacking in the education 
environment. For example, “People build a history of 
accomplishment in a company. They carry their 
reputations with them and non-performers eventually get 
their due. There are a who le host of inc entives bu ilt into 
the business environment that fosters a high level of 
team commitment,” 

An ABI Inform search on the broad term “student teams” 
over the past fifteen years produced 451 articles. Yet 
despite the face validity of Jones’ argument that student 
teams differ significantly from teams in the business 
environment, almost all of these articles used student 
teams as a surrogate for business teams and drew 
conclusions linked to business team performance. Only 
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six articles focused on learning how to make student 
teams more effective and recognized that differences in 
students versus experienced professionals, differences in 
work environment versus classroom, and differences in 
the incentives forbehavior make student teams a special 
case for researchers. This study examines priorresearch 
on student teams, including some of the research which 
uses student teams as surrogates for professionals. Then, 
a meta-analysis of this literature is performed, deriving 
a list of interventions which prior researchers have found 
to be effective with student teams. 

PRIOR RESEARCH 

Applying systems principles from Senge’s The Fifth 
Discipline, Jones (1996, 85) identifies systemic problems 
and solutions for student teamwork. The delay in 
feedback is a root cause of non-performance on student 
teams. Because “student teams do not have ready-made 
structures and procedures through which accountability 
and authority can function to quickly identify and correct 
poor performance” and because “it takes time to build 
consensus to deal with non-performance,” non- 
performers rarely get feedback within the severe time 
constraints of a quarter or semester. Therefore, non- 
performers rarely improve their performance. Jones 
seeks to develop “robust team systems” which will 
“promote patterns of behavior that support the goals of 
the education process.” 

One important structural component of a robust team 
system is the use of frequent, scheduled team meeting. 
According to Jones (1996, 85), “the fewer the meetings, 
the more delay or sluggishness in feedback.” The lack 
of feedback means that there is rarely an attempt “to get 
non-performers to do their share.” Frequent regularly 
scheduled meetings increase feedback and thus the 
opportunity to increase workload sharing. 

Jones’ (1996, 87) experience has been that “even a 
small amount of training [on working in teams] can 
produce dramatic increases in team effectiveness and 
efficiency.” Other quasi-experimental studies 
(Mennecke et al., 1998) have reached the same 
conclusion. Regular, frequent feedback from team 
members and the instructor are critical for effective 
student teams. 

Rapid and frequent feedback is the first principle for 
effective student teams according to Jones. This 



feedback is facilitated by frequent meetings and training 
on team process. This feedback pertains not only to the 
project content but to project team processes. The 
second principle is “individual accountability and 
responsibility [which] extends beyond the boundaries of 
the team” (Jones, 1 996, 87). Students receive individual 
and team grades, with many components of the project 
being graded. Thirdly, Jones ( 1 996,88) has found that 
studentteams must have “an early and high level of team 
member interaction.” He finds that “frequent team 
meetings” and “frequent small outputs from the team” 
increases feedback and allows performance problems 
with individual team members to surface more quickly 
and early in the academic term. 

When teams are diverse, “periodic process feedbackmay 
be a key” to effective teamwork (Watson et al., 1998, 
164). They find preconditions forteams becoming more 
instead of less effective over time: time for team 

discussion, feedback on both interpersonal and task 
performance, and linkage of individual to team 
performance. Although their research uses studentteams 
as a surrogate for industry teams in a study of team 
diversity, they used peer evaluations at midterm and at 
the end of the term. The feedback provided by peer 
evaluation “can remove students’ fear of having to 
choose between either low grade or having to do most of 
the group work (if other members fail to do their fair 
share” (Roebuck, 1998, 40). Students evaluated each 
other on both interpersonal and task dimensions. 
Instruction was also provided on effective teamwork 
topics. 

McKendall (2000, 281) has found that when team 
process is employed, students find thatworking inteams 
takes much longer than their previous experience with 
group work. Part of the difference may be because team 
projects without processes to ensure workload sharing 
result in a few students devoting time to the project 
while others have much less of a time investment. 
However, the team processes themselves take time. So 
Kendall incorporates meeting time into class time and 
assigns people to teams based on their schedules and 
ability to meet at the same times. However, she still 
finds that instructors who employ team processes must 
“be prepared for a lot of disgruntlement over the project 
grade” (McKendall, 2000, 281). Students believe that 
the time invested in the project makes them deserve an 
A. She cites the following problems with projects that 
do not deserve an A: 
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They chose a problem too quickly without fully 
investigating whether it is indeed a problem, 
they leapt too quickly to a solution, they did not 
interview a key person, they wrote their action 
plan but forgot to pace themselves and burned 
out before completion of theproject, they failed 
to confront those who were not producing . . . 
Given the large amounts of time and effort 
invested, however, they are not willing to see 
the errors reflected in their grade. 

Still, she (2000, 281) finds that “approximately two 
thirds of the teams report that their group worked better 
than any other group in which they had participated 
(some of them seem to attribute this to a happy fluke, 
which can be a bit frustrating) ” A reflective essay on 
theteam experience providessome evidence ofincreased 
self-awareness. Team process constraints employed by 
McKendall include a team contract, instruction in 
individual differences, collaboration, listening, problem 
solving, decision making, action plans, group roles and 
leadership, team communication patterns, conflict and 
cohesion. Each team presents its completed project and 
team members evaluate each other. 

Gardner and Korth (1998) find it important to address 
problems students have experienced with teamwork in 
the academic environment (mainly that a few students 
do all the work) and to stress the use of teams in the 
work environm ent. They assign stude nts to teams based 
on learning styles. 



Siciliano(1999) established team ground rules by having 
the class develop a profile ofan ideal team member. The 
profile consistedof four major components with detailed 
descriptors of those components. At midterm, each team 
member was evaluated by team peers using the four 
' major components and a scale from 1 to 5 (Siciliano, 
1999, 263): 

5 = always performed this way 
4 = often performed this way 
3 = occasionally performed this way 
2 = rarely performed in this way 
1 = never satisfied this criteria 

Average peer ratings were calculated for each student 
and shared with each student. Class averages and team 
averages were provided. Students were asked to develop 
a plan for improving their team performance. At the end 
of the term, the evaluation was repeated and became a 
grade component. Ratings were confidential for both 
administrations. Rajlich et al. (2000) found a 
statistically significant gap between how students 
perceive their performance on a team and how other 
students on that team perceive their performance. 
Although not empirically tested in this study, midterm 
evaluations shared with each student in summary form 
should work to lesson that gap and improve 
performance, as was Siciliano’s experience. Rajlich et 
al. were not focusing on student teams but using student 
teams as surrogates for professional application 
development teams. 



TABLE 1 

SICILIANO’S (1999, 263) IDEAL TEAM MEMBER CRITERIA 



1 . Do your part 


Complete the tasks assigned to you 




Be willing to put in the time necessary to complete your team assignment 


Ask if there is anything you can do 


Pull your own weight and do your share of the assignments 


2. Share your ideas 


Express your opinions 


Respond to other group members’ ideas 


3. Work toward agreement 
(consensus) 


Be open to other ideas, opinions, and perspectives 




Be willing to work together 


Work as a team (not solely on an individual basis) 


4. Keep a positive attitude 


Maintain a sense of humor 




Be courteous 


Give feedback in the form of constructive criticism 
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Prior researchers have found meeting management as 
well as meeting frequency to be important for effective 
team process. Shrage (1995) in his book, No More 
Teams!, a reaction to the misuse and mismanagement of 
teams in the work world, coined the term “meeting 
ecology,” emphasizing attention to factor promoting 
productivemeetings, hospitable meetings, andmeetings 
which generate commitment rather than apathy. Clark 
(1998) focuses his research on improving student team 
meetings. He begins his work with students by assigning 
readings on the importance of learning to work in teams 
and the importance of effective meetings for teamwork. 
He cites the estimated S37 billion (Kayser, 1990) lost 
annually in the United States due to poor meeting 
management. Studies have shown that people gain 
enthusiasm or commitment when they play apredefined 
role at a meeting. Clark (1998) cites the use of the scribe 
role or the sc hedule r role. Meetings are more productive 
if they are planned with agendas issued prior to the 
meeting. Meetings are more productive when each 
member has prepared priorto the meeting, often sharing 
documents with each other priorto the meeting. Finally, 
Clark finds that meetings are more productive when the 
focus is on an action items not information sharing. 
Clark finds that detailed meeting minutes andthe role of 
scribe are essential for productive meetings. They 
prevent rehashing old issues or debating decisions 
already made. 

McKeague et al. (1999) require weekly meetings and 
weekly reports in their marketing/engineering project, 
and they count these meetings with an accompanying 
weekly project report as 5% o f the course grade. Each 
meeting must also be recorded in a meeting journal. 
Each student on the team must keep this journal. A key 
finding of their study was that the grade on the project 
journals, used an independent variable in a regression 
analysis, is a significant predictor of the final project 
grade. During the term, student teams make five oral 
presentations of project progress to a faculty panel, so 
each student team receives frequent feedback before 
delivering the final written report. 

In a study of meeting agendas in the group support 
system context (Niederman and Volkema, 1999) found 
three benefits of employing agendas: improvement in 
the quality of meeting deliverables, increased group 
satisfaction with meeting results, and increased 
satisfaction with the overall process. These benefits 
were empirically tested using a survey of 238 group 
facilitators. 



Mennecke and Bradley ( 1 997) used predefined roles for 
team members and find that group cohesion is 
significantly improved. Other interventions, such as 
weekly meetings and frequentfeedback structures, were 
not employed in this study. Porter and Bryan (1996) 
find cohesion to be a problematic measure because it is 
not correlated with performance. Eighty project teams 
and 464 students participated in their study. Their 
findings indicate a moderating affect for the cohesion 
dimension related to performance. Cohesion has three 
dimensions: “comm itment to task, group attractiveness, 
and group pride” (Festinger, 1950). They focus on task 
commitment as a positive impact on student team 
performance. Instead, they find a direct impact comes 
from task process, “behaviors aimed at organizing 
members to get work done . . . setting goals, prioritizing 
work, developing workable plans that facilitate task 
accomplishment” (Porter and Bryan, 1996, 364). 
Although Porter and Bryan are using studentteams, they 
seek to empirically test theory concerning work team 
performance. Their “findings suggestthat management 
would do well to structure the team [task processes] so 
that [negative, unproductive] task conflict is minimized” 
(1996, 374). Mennecke et al. (1998, 1 10) subsequently 
added two interventions: “1) training on group process 
and 2) role assignment.” Again, they found higher 
cohesion and better project grades when roles were 
assigned compared to a control group. The highest 
cohesion and highest grades came from the teams where 
the added lessons on group process and roles were also 
employed. 

Stephens and Myers (2000) adapted the use of defined 
roles (Mennecke and Bradley, 1 997), removing the role 
of student liaison with the professor. Instead, they 
implemented structural interaction of the entire team 
with the professor throughout the term. Cohesion was 
not used as an outcome measure. They added a structure 
ofweekly team process constraints designed to solve key 
problems: procrastination and free loading (Stephens 
and Myers, 2000). Constraints included regularly 
scheduled weekly meetings preceded by an agenda and 
followed by minutes. A weekly action list was also 
required in which the distribution of specific work tasks 
among the team was identified by student andtime lined. 
Finally, they added peer evaluation with each team 
member rating the contribution of peers on each project 
component. Satisfaction with the team experience was 
significantly higher with the treatment group than with 
the control group who were simply assigned the project 
in teams with no role orprocess constraints applied. The 
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treatment group perceived the roles to be valuable and 
found the weekly meetings with action lists to be 
particularly valuable. 

INTERVENTIONS FOR 
EFFECTIVE STUDENT TEAMS 

Based upon this prior research, instructors may choose 
to exert control over task process through using 



interventions. Table 2 summarizes the interventions 
employed to increase the effectiveness of student teams. 

Instead of assigning ateam project and then grading the 
deliverable at the end of the term, the instructor may 
address workload sharing and thus learning for 
individual students providing rapid, frequent feedback. 
This feedback can be based on project deliverables, 
weekly meeting agendas, minutes, and action lists, 



TABLE 2 

META-ANALYSIS OF PRIOR RESEARCH 
INTERVENTIONS FOR IMPROVING EFFECTIVENESS OF STUDENT TEAMS 



Interventions 


Source(s) 


Rapid, frequent feed-back on team process 
(documentation in form of reports, minutes, 
agendas, action lists, meeting journals) and 
project content (reports, presentations, 
project deliverables) 


Jones (1996), Watson et al. (1998), McKeague et al. (1999) 


Frequent, regularly scheduled meetings 


Jones (1996), McKeague et al. (1999), Stephens and Myers (2000) 


Training on working in teams 


Jones (1996); M ennecke etal. (1998); McKendall (2000), Siciliano 
(1999) 


Individual as well as team accountability 


Jones (1996), McKeague et al. (1999) 


Peer evaluation 


Watson et al. (1998), Roebuck ( 1998), M cKendall (2000), Siciliano 
(1999), Rajlich et al. (2000), Stephens and Myers (2000) 


Team contract 


McKendall (2000) 


Reflection on team experience 


McKendall (2000) 


, Student evaluation of other team projects 


McKendall (2000) 


Lessons on importance of teamwork 


Gardner and Korth (1998) 


Discussions of prior problems with 
teamwork 


Gardner and Korth (1998), Siciliano (1999) 


Assign to teams based on learning styles 


Gardner and Korth (1998) 


Development of ideal team member profile 


Siciliano (1999) 


Attention to meeting management 


Clark (1998), Stephens and Myers (2000) 


Weekly scheduled meetings 


McKeague et al. (1999), Stephens and Myers (2000) 


Meeting time in class 


McKendall (2000) 


Defined roles in meetings 


Clark (1998), Mennecke and Bradley (1997), Stephens and Myers 
(2000) 


Use of meeting agendas 


Clark (1998), Stephens and Myers (2000) 


Use of meeting minutes, journals 


Clark (1998), McKeague et al. (1999), Stephens and Myers (2000) 


Team process as grade component 


Stephens and Myers (2000) 


Meeting management as grade component 


McKeague et al. (1999) 


Use of e-mail for information sharing prior 
to meetings 


Clark (1998), Stephens and Myers (2000) 


Use of action lists in meeting 


Stephens and Myers (2000) 
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meeting journals, milestone presentations, and midterm 
peer evaluations. Predefined roles for team members, 
training on team process, discussion about what makes 
for an ideal team member have all been shown to 
improve team performance. Linking the rapid, frequent 
feedback components to the project grade at an 
individual level helps to make individuals as well as 
teams accountable andpro vides external intervention for 
non-performers, the same kind of intervention a project 
manager would provide in the business environment. 

CONCLUSIONS AND FURTHER RESEARCH 

The interventions identified in the meta-analysis may be 
grouped by interventions appropriate for project 
initiation, for work on project deliverables, and for 
project closure . Table 3 provides a guide for the phased 
application of interventions shown to be effective 
through prior research. 



Much work remains to be done on increasing student 
team effectiveness. Business schools continue to 
employ student teams in many courses, and to 
emphasize the importance of teamwork. Certainly, 
teamwork is an integral part of an information 
systems majors’ education. Research supports the 
view that students do not learn to work in teams 
effectively by simply being given a team project or 
assignment. They may, in fact, learn the opposite 
from intended lessons. We need to establish a body of 
research on interventions for effective student teams, 
given the constraints of the academ ic environm ent, by 
tracking interventions employed and results achieved. 
Clearly, more empirical research is needed on student 
teams as student teams and not as surrogates for 
professionals in the business environment. 



TABLE 3 

PHASED APPLICATION OF INTERVENTIONS 



Project Phase 


Interventions 


Initiation 


Assign or allow to self-assign to teams based on schedules, profile of ideal team 
member, learning styles, personality type or some rational criteria. 


Discuss importance of teamwork, problems experienced with student teams, ways to 
resolve these problem s, meetings management. 


Teams formulate a contract which specifies criteria for non-performance. 


Require meetings early in the term (allowing some class time for meetings to observe 
interactions) for project identification, contract formulation. Frequent meetings early 
in the term help to identify low performers and influence their behavior. 


Work on Project 
Deliverables 


Required regularly scheduled weekly meetings with weekly reporting in the form of 
agendas, minutes, action lists, meeting journals. Allow some class time for meetings. 


Predefined roles in meetings and in project activities. 


Weekly grades based on meeting deliverables. 


Individual grades on some deliverables which should be prepared prior to the team 
meeting where the team version will be developed. 


Use of e-mail to facilitate information sharing. 


Periodic brief lessons on effective team process topics. 


Milestone presentations in which students evaluate other teams and receive feedback 
from the instructor (3 recommended). 


Mid-term peer evaluations with feedback summarized for each student. 


Closure 


Peer evaluation. 


Assessment of how well feedback was used. 


Assessment of team process. 


Assessment of project deliverables. 


Student reflection on behavior in the team, the team process, and the learning 
experience. 


Instructor reflection on team process effectiveness. 
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Extended Abstract 



Electronic commerce, or e-commerce, has changed the 
way we do business both domestically and 
internationally. According to a report released by 
Iconocast (2001), consumers will buy S74 billion worth 
of goods online in 2001 and that figure is expected to 
grow to $2.7 trillion by 2005. The world’s 60 million 
businesses will spend $6.3 trillion worldwide using 
business-to-business (B2B) Internet commerce by 2005, 
up from $3.4 billion in 2000. 

To prepare students for employment in the Internet 
economy, many universities havebegun to offerdegrees 
in electronic commerce in addition to their traditional 
information systems (IS) programs. However, e- 
commerce curriculum has varied significantly from 
program to program. 

In a 1999 survey conducted on e-commerce academic 
programs, it was found that the amount of e-commerce 
course offerings tripled between 1998 and 1999, and 
there were five times as many courses offered in 1999 as 
there was in 1997. The majority of the courses 
investigated, approximately sixty-one percent, could be 
defined as e-business overview or survey courses, which 



included various multidisciplinary concepts (Sendall, 
1999). 

As e-commerce degrees continue to proliferate, higher 
education will benefit from an understanding ofwhat set 
of skills the industry requires of recent graduates to be 
successful in the Internet economy. As that begins to 
take shape, industry will too benefit, as employers will 
be able to anticipate that graduates entering the 
workplace will have coherent e-commerce foundation 
knowledge. 

What are the skill sets required of graduating students 
seeking employment in the field of e-commerce? Do 
they differ from the skill sets required for employm ent in 
other areas of information systems? 

METHODOLOGY 

A study performedby Landry, Longenecker, Haigood& 
Feinstein (2000), compared variousinformation systems 
(IS) job types based on IS faculty perceptions of the 
skills that comprised each job type (See Appendix A). 
A “job type” is defined as an IS specialization area, 
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based on that specialization’s corresponding job in 
industry. A “skill depth” is used to describe the level of 
understanding of a learning objective, according to 
Bloom (1956). 

The authors believed that by “isolating the various skill 
depths for different IS jobs” the study would, “contribute 
to efforts toward defining a more specialized IS 
curriculum.” The study concluded, in part, that one of 
the objectives of future studies ought to be to determine 
what skill elements and career path s, if any, are currently 
missing in IS curriculum. 

Another study by Longenecker, Feinstein, Haigood and 
Landry (1999) collected one thousand ads from Internet 
publications in 17 major national newspapers. Each ad 
consisted of words or short phrases that were grouped 
and categorized under the exit curriculum sub-areas of 
the IS ’97 Curriculum Model for UndergraduateDegree 
Programs in Information Systems (1997). 

Employing the same methodology as in the study cited 
above, we will conduct a content analysis by word 
frequency counts using “e-commerce” as a modifier, 
and compare the results to the exit curriculum sub-areas, 
or tasks performed by IT professionals, as identified in 
the IS ’97 Curriculum Model. The panel will discuss if 
any skill elements are missing from IS curriculum that 
prepares students for careers specifically in e-commerce. 
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APPENDIX A 

SKILL DEPTHS ACROSS INFORMATION SYSTEMS JOB TYPES 



1. Individual and Team Interpersonal Skills 

1 . 1 Learning, Encouraging, Listening 

1.2 Being Organized 

1 .3 Comm itting/Completing W ork 

1.4 Ethics 

1.5 Writing 

1.6 Speaking 

1.7 Vision/Mission Development 

1.8 Self-Directed 

1.9 Leadership 

1.10 Time Management 

2. Business fundamentals 

2.1 Business Process and Functions 

2.2 Traditional responsibilities — Accounting inbound/outbound logistics and distribution, HR, Production, 
Marketing 

2.3 Business Relationships and Mechanisms 

3. Systems Analysis and Design — Strategic utilization of IT, systems theory/thinking, IS Planning, IS 
development, customer focus 

4. Software Development — Application development, client-server programming, problem solving 

5. Web Development — Website design and management, web programming 

6. Database — Modeling, design, procedures, administration 

7. Systems Integration — Hardware, Networking, LAN /WAN, OS, Management/Interoperability 

8. Project Management — Team Leading, coordination, scheduling/tracking, resource management, lifecycle 
management 
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WORKSHOP: 

INCORPORATING THE SOCIAL DIMENSION 
INTO THE E-BUSINESS CURRICULUM 
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Currently the rate of change in the business world 
exceeds human capacity to process that change. This 
includes virtually every facet of comm erce in every part 
of the globe. The internet and resulting e-business 
applications, wireless communication, broadband and 
the PC are only one set of components in this sweeping 
series of changes. Areas such as biotechnology, the 
Human Genome Project, bots and robotics add additional 
complexity and suggest that the rate of change may 
increase even more in the short term. All of this and 
more creates a workplace and a society unlike any we 
have lived through in the history of human beings. 

It is vital that we understand and seek to utilize all that 
technology can offer that will enhance both life and 
business. In the frantic pace that exists in the 
marketplace it appears that a “lemming-like “ rush is on 
to embrace the latest technology with little awareness or 
concern as to what impact these devicesand approaches 
may have on our lives as people. Failing to ask forward 
thinking, probing and difficult questions consigns us to 
allow unnecessary and unhealthy risk to enter both the 
business world and our personal lives as well. 

For example, does the practice in France of virtually 
everyone taking off six weeks for vacation suggest a 
country that risks missing out on the e commerce boom 
or a group of people who have thought out their values 
in critical areas more thoroughly than we in the West 
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have done? Do our 14-16 hour days really help us in the 
long run? What issues demand attention if we seek to 
plan long term work careers? Issues of how much is 
enough and the growing global disparity in wealth and 
resources require attention as well. These are some of 
the questions that will be explored in this seminar. 
Experience tells us that few if any take time to even 
reflect on these questions. 

This workshop will focus on long-term social 
implications of current and emerging technologies in the 
context of e-business. This is a perspective that few 
have addressed but one that is becoming increasingly 
important. Balancing the demands of personal and work 
life have always been difficult but especially so in 
today’s fast paced business environment. Humans 
cannot sustain the last paced speed at which decisions 
are being made. Being connected through e-mail and 
mobile phones 24/7 leaves little opportunity to com- 
pletely disconnect from work. Learning to balance all 
these demands with family responsibilities is a challenge 
for most. Businesses who have addressed these issues 
have experienced positive results. 

Social implications go well beyond the individual and 
personal domain. The digital divide between the haves 
and have nots continues to widen globally. As the world 
becomes smaller and more connected, cultural barriers 
will become less pronounced. The networke d electronic 
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world of the future holds many potential social impli- 
cations that should be addressed proactively today. 

In addition, as educators, we need to train the next gen- 
eration of businesspersons as those capable of leading 
others into places of balance and connection. How can 
such issues be raised in classes where the drive seems to 
focus solely on how much money can I make? It is 
imperative that we raise these questions and incorporate 
a discussion of these issues in our current curriculum. 

Participants will actively reflect on these and otherissues 
during the workshop session. They will be exposed to 
the emerging role of the relationship coach as one way 
of addressing theseissues from the business perspective. 

Overall Objectives 

• Heighten the awareness of these emerging social 
issues 



• Identify the questions and issues we should raise 

• Recognize the importance of incorporatingthis dimen- 
sion into the current e-business curriculum 

Major Issues to be Addressed 

• Work/life balance 

• The Digital Divide — Global Implications 

• Technological advances in biotechnology, nanotech- 
nology and information technology — their combined 
influence on society 

• Robotics technology and its implications for future 
generations 

• Influence of these trends on business school 
curriculum 
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FACTORS AFFECTING SYSTEMS 
IMPLEMENTATION: THE CASE OF 
ENTERPRISE RESOURCE PLANNING SYSTEMS 
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INTRODUCTION 

A crucial topic in Information Systems curricula is the 
implementation ofsystems. Systems analysis and design 
is a required course in most IS programs. One of the 
most important lessons in a systems design course, or 
even in ageneral Information Systemscourse, is that all 
systems are not always successfully implemented. 

Organizationsadopt information technologiesto remain 
competitive, to raise productivity, and to improve 
decision making (Alavi and Joachimsthaler, 1992). In 
early studies of technological innovations, it was 
assumed that once a technology had been adopted, 
implementation would follow more or less auto- 
matically. Yet approximately 40% of all Information 
Systemsprojects fail or are cancelled (Silverman, 2000). 
Information technologies have the potential to change 
entire industries. But although the capabilities of these 
technologies become more and more sophisticated, the 
technologies themselves are often under utilized or do 
not meet their potential. 

FACTORS AFFECTING IMPLEMENTATION 

There are a number of factors affecting implementation 
success and failure. The larger the project, the greater the 
risk. When a system involves many different facets of 
the organization, it becomes more difficult to manage, 
coordinate and plan for. The larger the system, the more 
things that can go wrong. Testing is another factor. 
Testing is essential with any system no matter how large 
orsmall. But when a system is complex, testing becomes 
much more complicated. Each module itself needs to be 
tested, as well as the integration of different modules. 



and in some instances the integration of the entire system 
with other external systems with which it will interact 
must also be tested. Adequate testing ofsuch large scale, 
complex systems are expensive and take time. This is 
related to another factor which affects system success 
and failure — the difficulty of estimating the time and 
cost it takes to implement large scale, complex systems. 
Companies are often not willing, or able, to devote the 
time needed for successful implementation. Hence, many 
systems fail after going live too early. 

Today, most software is not developed from the ground 
up. Existing so ftware is usually customized for specific 
companies' needs. This, too, can cause all sorts of 
implementation problems, especially with large, 
complex software. 

This paper focuses on applying these concepts of 
information systems success an d failure to a technology 
that over the last five years has been widely adopted 
world-wide. 

ENTERPRISE RESOURCE 
PLANNING SYSTEMS— BACKGROUND 

Enterprise Resource Planning (ERP) systems were 
developed in the 1990s to providea comprehensive and 
integrated means for the management and operation of 
an enterprise. ERP systems generally automate and 
integrate the majority of business processes and share 
common data and practices across the enterprise. 

The scope of ERP encompasses nearly every aspect of an 
enterprise, but is often categorized in three primary 
applications: manufacturing and logistics, financing and 
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accounting, and human resources and payroll. ERP 
allows a real-time summ ary of all the operations within 
the enterprise (Grad, 1999). 

It is only in recent years that computer and software 
technologies have been able to support such a 
comprehensive platform for business operation. It is not 
surprising that ERP has evolved from more primitive 
packages that made use of technology available at the 
time. These earlier software applications resulted in a 
clutter of various technologies customized for each 
department that worked effectively within the 
department, but could not really interact with other 
departments (Slater, 1999). A need arose for an 
integratedsystem that could seamlessly transfer relevant 
information between the various departments across the 
entire enterprise. This was the beginning of Enterprise 
Resource Planning. 

The adoption o f ERP initially gained momentum through 
Y2K compliance. Rather than upgrading existing 
software, many companies moved to the new ERP 
systemsthat were already Y2K compliant (Richardson, 
1999). However, ERP has the potential to bring many 
additional and substantial benefits to companies 
including standardization of computer systems and 
platform, integration of heretofore disparate business 
processes and elimination of redundancies within the 
enterprise, leading to a more efficient and more 
competitive enterprise. ERP also enables globalizationof 
the enterprise by supporting multi-language and multi- 
currency environments. Different accounting and legal 
practices can also be automatically taken into account. 
For companies in the 1990s seeking growth and 
globalization, this ability to plan and allocate all 
resources across the enterprise was just whatwas needed 
(Slater, 1999). 

ERP IMPLEMENTATION PROBLEMS 

Unfortunately, the comprehensive nature ofERP is also 
the root of its greatest disadvantages. Since ERP 
integrates operations from the lowest levels of an 
enterprise to the very top, the implementation of ERP 
often requires not only a complete reorganization of the 
company’s IT resources, but fundamental changes in 
corporate culture, business processesand management as 
well (Schneider, 1999). As a result, implementing ERP 
has become notorious as a long, costly and painful 
process, with several high-profile cases of companies 
canceling their ERP implementations midway after 
encountering too much difficulty. 



One particularly high profile case involved Hershey 
Foods Corporation. In the Fall of 1999, Hershey, the 
nation’s largest candy maker, was not able to make 
shipments for Hallowe’en because of the implementation 
of an SAP based integrated ERP system. This case 
illustrates many of the factors affecting implementation 
success discussed earlier. The system was large and 
complex, costing $110 million. Three different software 
and computer systems from SAP AG, Manugistics 
Group, Inc. and Siebel Systems, Inc. had to work 
together. In such a situation, it is difficult to apportion 
blame when something goes wrong. SAP said the 
problems were due to Hershey going on line too early 
and with a huge piece of it all at once. Hershey blamed 
the problems on the integration of two different systems. 
A newly developed system meant to ease shipping and 
logisticsbetween Hershey plants and retailers was to be 
integrated with an existing nationwide order-taking and 
distribution system. According to Hershey, this caused 
major disruptions. They said the integration of the two 
systems had not been tested adequately. The problems 
continued past Hallowe’en, through Christmas and 
Valentine’s day. 

This complex and difficult implementation process is 
one of the greatest issues facing ERP today. One major 
problem is the coordination of technical and business 
expertise. Consultants hired to implement an ERP 
solution may be experts on the software and technology 
but cannot be expected to know very much about the 
minute details of the business operations within the 
company. On the other hand, the companies that 
implement ERP are often ignorant of the extent and 
complexity of effective ERP solutions (Deutsch, 1998). 

A good illustration of this is another high profile case 
involving Whirlpool Corp. Whirlpool had to delay 
shipme nts of appliances to distributors and retailers. As 
a result of this delay, Home Depot Inc. stopped ordering 
appliances from Whirlpool, at least temporarily. 
Whirlpool blamed the delay on a newly installed ERP 
system. But SAP said they warned Whirlpool before its 
system went on line that it had problems generating 
some supply data fast enough. According to SAP, 
Whirlpool ignored the cautionary advice and chose to go 
live anyway. 

ERP SUCCESS FACTORS 

A study by Benchmarking Partners has identified “Ten 
Critical Success Factors for ERP Implementation” in 
which priority, a dedicated team, and clearly defined 
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business and strategic objectives are identified as the 
most critical factors. Senior management, as a top 
priority for the company, must take the initiative, with 
appropriate management bonuses and incentives for 
successful implementation. Team members must be 
completely dedicated to the project and removed from 
other responsibilities, with preferably at least one 
sponsor in senior management. Finally, the company 
must identify specific business objectives and plan a 
phased deployment with repeated tested at each stage, to 
ensure that things are going in the right direction 
(Benchmarking Partners, 1999). Other tips identified by 
similar studies include concurrentemployment training 
for the software with continuous feedback from the end 
users who will actually use the system and minimum 
alteration of the standard software package so that future 
upgrades are easier (Deutsch, 1998; Slater, 1998). 

CONCLUSIONS 

ERP represents the culmination of three decades of 
business computer technology with increasing 
capabilitiesandcomplexity. The technologyhas reached 
a point where fundamental changes in business methods, 
strategy and culture are needed. Consulting firms have 
often concluded that failure of ERP is often the result of 
insufficient planning, awareness, vision and training, not 
the result of any fundamental flaws in the ERP 
technology itself. Indeed, the study by Benchmarking 
Partners concludes that success has almost nothing to do 
with technology and almost everything to do with p eople 
and process. The future of successful ERP 
implementation does not rely on further improvements 
oftechnology, buton bringing people and business up to 
speed on the appropriate use of ERP technology to fit 
their defined business needs and objectives. 

The realization that ERP is not a turnkey solution is the 
first step towards not only solving the implementation 
problem, but also wider applicability of ERP in different 
vertical segments. As businesses learn what they want 
from ERP and to successfully incorporate it into their 



operations and planning, ERP providers will in turn 

improve their ability to accommodate differentindustries 

and business models. 
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OPPORTUNITIES FOR APPLIED 
RESEARCH, PRACTICAL KNOWLEDGE, 
PROFESSIONAL NETWORK, AND 
EXTERNAL FINANCIAL OUTREACH? 

Rick Weible 
Marshall University 



ABSTRACT 

Interested in Improving: Your Teaching, Student Placement, Opportunities for Applied Research, Practical 
Knowledge, ProfessionalNetwork and External Financial Out Reach? Try a Faculty Internship. It is a win-win-win- 
win opportunity. You win, the faculty member gets many benefits. The company wins, they gain access to your 
expertise and an inside track for recruiting your best students. Students win , they receive better lectures, great guest 
speakers, and better advise about career choices. The school wins, you promote your institution by your work, and 
your school has an inside track on seeking financial or other grants from companies. 



INTRODUCTION 

One criticism that is often level at the academic 
community by business leaders is that colleges of 
business have fallen behind the times and are not “in 
touch” with current developments (Smith, 1995). In the 
technology areas some times the opposite accusation is 
leveled, colleges are investigating the latest and greatest 
technologies that are ahead of business adoption. In 
marketing a common perception among practitioners is 
that theories emanates from the “ivory tower” of 
academia without a foundation in practice (Nataraajan, 
Henthorne and LaTour, 1998). Many of these percep- 
tions arise due to the gap between business practices and 
colleges of business principles. 

Many faculty, especially new faculty lack practical 
experience in there academic field. They have taken the 
courses that survey their field. They understand all the 
principles and theories, but lack experience in applying 
all this knowledge. The common route to this 



experience in business is consulting. But consulting is 
somewhat of a catch 22 situation, as you need 
experience to get the consulting work. If you are lucky 
you have a faculty colleague willing to share consulting 
work to help you gain experience. Another approach is 
a Faculty Internship. Much like a student internship you 
spend a week, a semester, or the summer working in an 
organization. This approach has many benefits and few 
drawbacks. The benefits come to you, the company you 
work for, as well as to your students and your school. 

MY EXPERIENCE AS A FACULTY INTERN 

As I faced the dilemma of education, but no experience 
and being in a small IS department, I realized an 
internship was a perfect solutions. I could gain some 
experience, without other faculty assistance. Timing was 
also on my side. I had just been approached by an IS 
manager at one of the few companies with a major IS 
shop in my area (Columbia Gas Transmission) about 
student internships. When I suggested the idea of me as 
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a faculty intern, he was at first hesitant, but talked it 
over with his company management and I was in. 

There were a few problems on the company side, such 
as hiring me as an independent contractor. This was 
solved by going through a local temporary employment 
agency. I was teaching summer classes, so I need to be 
able to work around my summer schedule, but this 
turned out not to be a real problem. 

My assignment was to be a project manager on the 
installation of a new software package for tracking 
employee software skills and company position needs. 
The product they purchased is called Skills View from 
a company in New England. The projectrequired coor- 
dination from several departments in the IS side of the 
business. There were several related projects occurring 
that were interrelated to the Skills View project, so I had 
the opportunity to work with several department 
managers and get an in depth look at what they were 
doing and how they operated. 

Several students from my MIS program were hired as 
interns, so it was good to have a close look at what they 
were involved in doing and the skills they were gaining. 
Most of those interns were offered positions when they 
graduated. So the student internship program we started 
worked very well. 

I worked most of May and June, then taught all of July 
and into August. I had only a couple of week available 
at the end of summer to finish the project. It would have 
been better to have a continuous block and not had to 
break it up with teaching. 

Overall I would rate my experience as very successful. 
But there are some thing I would do differently based on 
my experience and from working with the consultants 
from Skills View. First, I would have clearly defined 
the desired outcomes for the project I was working on. 
I generally knew what was wanted, but specific 
objectives would have given me more confidence in 
being successful. This was specially true since I had 
little experience and no one to mentor or advise me. I 
would have developed some milestone objectives and a 
project schedule. Finally, I would have had weekly 
evaluation sessions with the manager I was working 
with, learning more from his experience. 

LENGTH OF INTERNSHIP 

While my experience lasted most of the summer, 



Friedlob and Trapnell (1988) suggest a Mirco-Intership 
that last for a few days to a week. So the length may not 
be the most important factor. Friedlob and Trapnell 
(1988) suggest several advantages to their micro- 
internship. 

• Cost of faculty member is reduced 

• Opportunity to maximize exposer to more areas of the 
company quickly 

• Lower cost to the company 

• Hands-on experience can be available to all faculty 

SUCCESS FACTOR 

Carnes and Gierlasinski (1999) have six suggestions for 
a company considering a faculty internship program. 
But the suggestions will work for faculty member as 
well. 

1. Begin on a small scale. Make contact with a local 
company and see what they need. 

2. Match the faculty member’s talents with needs of 
the job. Do they need a special project consultant or 
a part-time regular employee? Will the intern work 
with clients or just staff? Do training? Both parties 
should put the internship into written form. 

3. Integrate the intern into the business in a 
professional manner. Dress code, former students, 
work hours, etc all need to be detailed. 

4. Be flexible in compensation issues. Pay what the 
project is worth. 

5. Conduct a post-internship evaluation. Both intern 
and company can learn from an objective exit 
interview. 

While this list in not exhaustive, it is a good starting 
place. Work from here to custom a program to meet 
your situation and needs. 

BENEFITS 

Who benefits from faculty internships? Everyone. It 
can be a win-win-win-win opportunity. Faculty gain 
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insights into current business practices, companies gain 
access to the professor knowledge and an inside track on 
students, students have more relevant lectures and an 
inside track to employers, and college/university win 
with an inside track on funding opportunities. 

FACULTY MEMBERS 

Faculty member can see the theories and principles they 
teach in actions, meet new people and expand then- 
knowledge. Individual benefits include: 

1. The opportunity to gain practical experience by 
working side-by-side with practitioners; 

2. The introduction to problems business face that can 
lead to researdi ideas; 

3. An expanded network of professional contacts; 

4. Gain a greater understanding to the skills needed to 
taught in your curriculum; and 

5. Extra income. 

As a faculty intern I gained all of these benefits. 

COMPANIES 

The firms involved have many benefits. The benefits to 
the firm include: 

1. Access to the expertise of academics with current 
knowledge of the latest technologies and new 
applications for other technologies. 

2. Access to a continuing supply of better educated job 
candidates and ongoing interactions with potential 
employees. 

3. The use of your company in teaching examples. 

4. The opportunity to send guest speakers to promote 
the firm. 

Columbia Gas Transmission has found our students to 
make excellent employee and actively seeks them. 



STUDENTS 

Students are the big winner in faculty internships. The 
benefits to your students are: 

1. Lectures filled with practical examples from local 
companies, 

2. Guest speakers with first hand industry knowledge, 

3. Faculty contacts that lead to great jobs, and 

4. Better advise in career choices. 

I feel my students are better prepared and have access to 
better jobs because of faculty internships. 

THE INSTITUTION 

The school wins. Once a firm has a relationship with 
your school they will be sympathetic to your financial 
needs, such as student scholarships, equipment grants, 
or other financial grants to your school. 

ColumbiaGas Transmission has donated over forty PC’s 
to my college of business. 

CONCLUSIONS 

Faculty internship may not be for everyone, but they are 
an excellent way for new faculty to gain valuable 
experience. Companies enjoy a closer tie to the 
university and new employees. Students have 
professors with practical knowledge. Institutions have 
access to outside funding that may not other wise be 
available. 
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DEVELOPING A ROBUST SYSTEM FOR 
EFFECTIVE TEAMWORK ON LENGTHY, 
COMPLEX TASKS: AN EMPIRICAL 
EXPLORATION OF INTERVENTIONS TO 
INCREASE TEAM EFFECTIVENESS 



Charlotte S. Stephens 
Louisiana Tech University 

Martha E. Myers 
Kennesaw State University 



The synergy of informed minds working together is incredible; whereas unprepared minds working together 
is nothing more than pooled ignorance (Jones, 1996, 87). 

ABSTRACT 

Management of student teams in information systems courses so that students learn how to participate in teams 
effectively is an important task for IS professors. However, most research on this topic applies what is learned from 
student teams to teams in the work world, not to the academic en vironment. Three professors at two universities in 
six classes apply interventions to improve student team process in two courses: Database Management Systems, 
Systems Analysis and Design. Two control groups were used. Results indicate that these interventions make a 
significant difference, although caution must be used in interpreting the results of this exploration. 



INTRODUCTION 

Because of the increasing emphasis on teamwork by 
businesses and subsequently, by higher education, 
teaching students to work effectively in project teams is 
an important issue (Gardner and Korth, 1 998; McKeague 
et al, 1999). However, the characteristics of student 
teams limit the applicability of research on team or 
groupprocess outside the academic environment (Jones, 
1996). A broad ABI Inform search conducted for the 
most current fifteen years using the search term “student 
teams listed over 450 articles. However, only six 
articles during this fifteen-yearperiod dealt with making 
student teams in an academic environment more 
effective. Most research used student teams as a 
surrogate for professional work groups, despite the fact 



that most student teams operate in a very different 
context and with a very different level of work 
experience and domain expertise. A meta-analysis of the 
literature on effective student teams was conducted to 
derive a set of interventions with student teams which 
had improved project team process and/or content 
learning (Stephens, 2001). 

Our interest in ways to make student teams more 
effective in an academic environment, particularly in 
information systems classes with term-long projects, 
arose from our suspicion that teamwork as practiced in 
academia was serving an opposite from intended 
purpose. That is, students were learning how not to 
participate in teams effectively. Students tended to 
procrastinate until the last few weeks of the term and 
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then with the high stress levels experienced then, 
irritability and conflict in teams increased. Frequently, 
high achieving students on the team exerted a heroic 
effort (“all nighter ) to complete theproject either alone 
or in pairs. Other students got a free ride and the same 
grade. The team experience seemed to be decreasing 
learning and increasing grade inflation as all students 
received grades earned by the better students. Feedback 
on the project usually came at the end of the term, but no 
action was required as a result of this feedback. Instead 
of learning how to execute a project effectively as part of 
a team, students seemed to be learning all the “what not 
to do’s for effective teamwork. In fact, we asked 
ourselves, “would the students enter the work world be 
better prepared for effective teamwork if they had not 
had teamwork in the academic environment? 

Jones (1996) calls for the development of a robust 
system for managing student teams, a set of 
interventions that is not instructor dependent or task 
dependent as long as the task is a complex one like a 
term-long systems project. Preliminary results indicate 
that significant improvements can be made with these 
interventions and that the results are not dependent on a 
particular instructor. Thus, we demonstrate progress 
toward a robust system for effective student teamwork 
on lengthy, complex tasks such as a systems project. 

This paper describes our exploration of guidelines for 
increasing the effectiveness of student project teams, 
both in terms of learning course content and learning to 
work well in teams. First, we review the results of a 
meta-analysis of prior research (Stephens, 2001) and 
then we look at our exploration of inter vention sets with 
results obtained thus far (data collection and analysis for 
three of seven classes is incomplete at this writing). 
Student surveys and comments, peer evaluations, 
anecdotal information, and instructorjudgment are used 
for our results analysis. Finally, we make 
recommendations for interventions based upon prior 
research and our results. Needs for further research are 
addressed in our conclusion. 

RESEARCH QUESTIONS 

Our primary concern is with large, more complex 
projects that require coordination throughout the 
academic term. Other than projects, two types of 



assignments tend to be team activities in information 
technology courses (Stephens and O’Hara, 1999): 
research papers and case study analyses. This research 
does not address these two types of team assignments. 
We are concerned with tasks, like information systems 
projects, where groups have been shown to perform 
better than even the best individual in that group: “the 
problem has multiple parts, no one member has all the 
informationnecessary,the problem is at least moderately 
complex, interdependence is necessary, and there is 
enough time for members to process information 
(Watson, Johnson, Merritt, 1998, 162). 

Given projects with these attributes, what interventions 
will facilitate the following objectives? 

increase learning of course content 

learn how to work in teams effectively including 
professional meeting conduct, conflict resolution, peer 
evaluation, workload sharing, incremental versus last 
minute development, management of non-performers 

improve quality for final project deli verables 

increase student and instructor satisfaction with the 
team experience and the project deliverables 

prepare students to participate effectively on project 
teams in the work environment 

We begin our exploration of these questions knowing 
that the development of these guidelines will be an on- 
going process, one of the “holy grails of our 
pedagogical efforts. However, work over the past two 
years has been productive. 

INTERVENTIONS EMPLOYED 

A review of prior research and an analysis of 
interventions shown to be effective (Stephens, 2001) 
provides a background for instructor selection of both 
interventions and tim ing of interventions. 

Tables 2 and 3 provide summary data concerning the 
classes in which interventions were employed and the 
intervention sets employed in each class for this 
empirical study. 
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TABLE 1 

META-ANALYSIS OF PRIOR RESEARCH 
INTERVENTIONS FOR IMPROVING EFFECTIVENESS OF STUDENT TEAMS 



Interventions 


Source(s) 


Rapid, frequent feedback on team process 
(documentation in form of reports, minutes, 
agendas, action lists, meeting journals) and 
project content (reports, presentations, project 
deliverables) 


Jones (1996); Watson et al. (1998), McKeague et al. (1999) 


Frequent, regularly scheduled meetings 


Jones (1996), McKeague et al. (1999), Stephens and Myers (2000) 


Training on working in teams 


Jones (1996); Mennecke et al. (1998); McKendall (2000), 
Siciliano (1999) 


Individual as well as team accountability 


Jones (1996), McKeague et al. (1999) 


Peer evaluation 


Watson etal. (1998), Roebuck(l 998), McKendall (2000), Siciliano 
(1999), Rajlich et al. (2000), Stephens and Myers (2000) 


Team contract 


McKendall (2000) 


Reflection on team experience 


McKendall (2000) 


Student evaluation of other team projects 


McKendall (2000) 


Lessons on importance of teamwork 


Gardner and Korth (1998) 


Discussions of prior problems with teamwork 


Gardner and Korth (1998), Siciliano (1999) 


Assign to teams based on learning styles 


Gardner and Korth (1998) 


Development of ideal team member profile 


Siciliano (1999) 


Attention to meeting management 


Clark (1998), Stephens and Myers (2000) 


Weekly scheduled meetings 


McKeague et al. (1999), Stephens and Myers (2000) 


Meeting time in class 


McKendall (2000) 


Defined roles in meetings 


Clark (1998), Mennecke and Bradley (1 997), Stephens and Myers 
(2000) 


Use of meeting agendas 


Clark (1998), Stephens and Myers (2000) 


Useofmeeting minutesjournals 


Clark (1998), McKeague et al. (1999), Stephens and Myers (2000) 


Team process as grade component 


Stephens and Myers (2000) 


Meeting management as grade component 


McKeague et al. (1999) 


Use of e-mail for information sharing prior to 
meetings 


Clark (1998) , Stephens and Myers (2000) 


Use of action lists in meeting 


Stephens and Myers (2000) 



TABLE 2 

CLASS DESCRIPTORS 



Class 


Instructor 


Course 


University 


A 


1 


Database Management 


1 


B 


2 


Database Management 


1 


C 


1 


Analysis and Design 


2 


D 


1 


Analysis and Design 


2 


E 


3 


Analysis and Design 


2 


F 


2 


Database Management 


1 


G 


3 


Database Management 


1 
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TABLE 3 

INTERVENTIONS EMPLOYED IN EACH CLASS 



Interventions 


A 


B 


c 


D 


E 


* 

F 


* 

G 


Rapid, frequent feedback on team process (documentation in form of reports, 
minutes, agendas, action lists, meeting journals) and project content (reports, 
presentations, project deliverables) 






X 


X 


X 


X 


X 


Frequent, regularly scheduled meetings 


X 




X 


X 


X 


X 




Training on working in teams 








X 


X 






Individual as well as team accountability 








X 


X 






Peer evaluation 


X 




X 


X 


X 


X 


X 


Team contract 






X 


X 


X 






Reflection on team experience 








X 


X 






Student evaluation of other team projects 






X 


X 


X 






Lessons on importance of teamwork 








X 


X 






Discussions of prior problems with teamwork 


X 




X 


X 


X 






Assign to teams based on learning styles 
















Development of ideal team member profile 












X 


X 


Attention to meeting management 


X 




X 


X 


X 


X 




Weekly scheduled meetings 


X 




X 


X 


X 


X 




Meeting time in class 








X 


X 


X 


X 


Defined roles in meetings 


X 




X 


X 


X 


X 




Use of meeting agendas 


X 




X 


X 


X 


X 




Use of meeting minutes, journals 


X 




X 


X 


X 


X 


X 


Team process as grade component 






X 


X 


X 


X 


X 


Meeting management as grade component 








X 


X 






Use of e-mail for information sharing prior to meetings 


X 




X 


X 


X 


X 




Use of action lists in meeting 


X 




X 


X 


X 


X 





* In Classes F and G, students were allowed to choose to use no interventions or to use the interventions described on 
a web site. Student teams who chose to use the interventions comprise Class F. Students who chose to use no 
interventions comprise Class G. The same survey was administered to both groups. 



For the continuation of our study (Stephens and Myers, 
2000) with Class C, three major changes were made to 
the treatment for Class A: roles could be assigned for 
longer durations, a contract was required, and frequent 
feedback was emphasized with three team presentations 
instead of the prior final presentation of the project. In 
Class A, we had required thatroles rotate, allowing each 
team member to play each role. Students in this class 
(Stephens and Myers, 2000) asked that each team 
member play each of the three roles (Facilitator, Scribe, 
Scheduler) then that the roles be assigned for at least 
three weeks to the same student (see Stephens and 
Myers, 2000, fora full description ofthese roles). So we 



allowed students to take on a role for longerperiods of 
time. Secondly, we required a team contract. The 
contract must specify 

time and place of the regular weekly meeting; 

number ofpermissible absences andjustifiable reasons 
for absence; 

conduct during the meeting; 

conditions under which a team member could be 
terminated from the team; 
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grade to which the team aspired and time/week willing 
to devote to the project; 

use of e-mail (forexample,checke-mail at least once per 
day, use a certain format for attachments); and 

other issues ofimportance to the group. 

We emphasized that teamwork is important in our field 
and also emphasized that we were expecting professional 
team management, including the termination of non- 
performing team members. Schedules were thebasis for 
team assignment. Each class member introduced 
her/him self to the class and indicated when s/he could 
meet every week. Team contracts had to be accepted by 
all team members and by the instructor. Elements of the 
ideal team member (Sciliano, 1999) were used for the 
contracts. 

Thirdly, instead of one project presentation, Class C 
made three presentations. At each presentation, the class 
served as Steering Committee, completing anevaluation 
form and making recommendations. Teams submitted 
the project notebook for instructor review. Feedback 
given became a part of project documentation. Before 
the next presentation and review, changes recommended 
by the instructor should be made. These changes 
becomepart ofthe nextreview. One analysis and design 
class, Class C, used these team process constraints. 

The following term, three modifications were made for 
Class D. 

Additionalincentives were addedfor weekly meetings 
and for responding to feedback. 

Peer evaluation concerning the perceived behavior of 
each team member was administered at midterm. 

Some class time was allotted for team meetings. 

Points were given for agendas issued two days prior to 
the meeting and for minutes distributed within two days 
after the meeting. A template was used for the agenda 
and the minutes. Teams might not receive full points if 
minutes were sketchy or agenda items missing. Any 
student missing the meeting received no points for either 
the agenda or minutes, thus extending controls beyond 
the team itself. To encourage timely response to feed- 
back, the team’s track record on looping back to revise 
or correct project work based on feedback became a 
component of the project grade. Teams that had not 



responded to feedback on the first presentation by the 
second presentation had to meet with the instructor. At 
midterm, each team member completed a survey on the 
behaviors of other team members. These were 
summarized and given to each team member. Further, 
after midterm, teams were allowed some class time for 
meeting. Class D, an analysis and design class, used 
these additional conditions. 

Class E used almost the same conditions (no midterm 
peer evaluation and two versus three milestone 
presentations) as Class D but had a larger team size on 
average and had a different instructor. As withClass D, 
the course content was analysis and design. Classes F 
and G, database management classes, were given a 
choice concerning team process. They could use the 
guidelines posted on the class web site or use very few 
guidelines. The guidelines for Class G are most like 
those for Class A with the following exceptions. 

Class time was allowed for meeting 

Assignment to teams is based on studentpreference of 

three methods: no preference (random used), 

description of ideal team member, list of preferred 

team members 

Feedback in the form of two midterm deliverables 

Those students in the two classes who did not choose to 
use the team process guidelines were called Class G. 
They did participate in the feedback in the form of two 
midterm deliverables, class time allowed for meetings, 
and choosing their preferred method of team assignment. 
These two intervention sets are especially interesting 
because the class had both computer science and 
information systemsmajors. Additional data for analysis 
is provided in terms of performance according to choice 
concerning team process and choice according to major. 
Furthermore, these classes had a mix of non-traditional 
and traditional students. Many non-traditional students 
had work experience with teams, whereas traditional age 
students lacked such experience. 

The same instructor managed the team process in 
Classes A, C, and D. Another instructor managed the 
team process in Classes B, F, and G. A third instructor 
managed the team process in Class E. Instructor 1 
managed teams at both University 1 and 2 while 
Instructor 2 managed teams at University 1 and 
Instructor 3, at University 2. 
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RESULTS AND DISCUSSION 



As Mennecke and Bradley noted (1997), studies of these 
types will always have confounded results and are by 
nature, more exploration than experiment. We examined 
each of our research questions objectives in terms of 
both survey results, student comments, and professor 
judgment. Our objectives for these interventions 
followed in parenthesis with the survey question 
numbers (Appendix A) which address that objective are 
as follows. 

increase learning of course content (1 8,1 9, comments, 
professor judgement) 

learn how to work in teams effectively including 
professional meeting conduct, conflict resolution, peer 
evaluation, workload sharing, incremental versus last 
minute development, management of non-performers 
(1-17, 20-26, comments, professor judgment) 

improve quality for final project deliverables (7, 
comments, professor judgement) 

increase student and instructor satisfaction with the 
team experience and the project deliverables (27,28, 
comments, professor judgment) 

prepare students to participate effectively on project 
teams in the work environment (professor judgment) 

Increased Learning 

All student groups agreed that the project helped them to 
understand course topics, with the strongest agreement 
being from students who had points given for weekly 
team meetings (Appendix B). Surprisingly, students 
were indifferent about whether the project helped with 
exams except for students who had grade incentives for 
weekly meetings (D and E) and they agreed that the 
project helped them with the exams. In fact, the most 
frequent rating for that statement was a mode of 5 or 
strongly agree between those two groups. The 
interventionthat made a difference with learningcourse 
content was to provide a grade incentive for weekly 
meetings, as demonstrated by meeting minutes and 
action lists submitted weekly. Professors involved also 
judged that background readings were completed on a 
week-to-week because the readings were needed to 
complete incremental project deliverables. 



Working in T earns E ffectively 

The use of agendas, meeting minutes templates, action 
lists, and designated roles all contributedto professional 
meeting conduct. All students, including the control and 
those who chose not to use constraints, agreed that they 
learned to use meeting time more effectively as a result 
of the project. All but one of the treatment groups 
agreed that agendas were valuable and contributed to 
effective use of meeting time. All groups agreed that 
action lists were valuable and contributed to equitable 
workloads. The strongest agreement came from the 
groups that were rewarded for weekly meetings (D,E). 
Peer evaluations at midterm seemed to be an effective 
intervention. Group D had a more favorable experience 
overall than other groups and was the only one with this 
intervention. Group D also dismissed one student from 
a team for contract violation. The effect of midterm 
evaluation needs further exploration based on these 
results. Peer evaluationsat the endof the quarter did not 
result in grade changes for any individual student 
because they indicated that individual contributions had 
been reasonably equitable. Professors judge that action 
list monitoring, contracts, and weekly meetings 
positively affect workl oad sh aring. Results on the use of 
roles are mixed. The facilitator role was found most 
valuable andteams with regular weekly meetings agreed 
thatthe scribe and scheduler were valuable. The weaker 
results for the scheduler may indicate that more class 
time needs to be devoted to this role’s duties. 

One of the most important results of the interventions 
experienced by groups C,D, and E was the requirement 
that deliverables be presented throughout the quarter. 
Results indicate that an incentive for weekly meetings 
and increased use of feedback along with the use of 
rolesand meeting management, improves the projectand 
the project experience (Class D,E,A). The deliverables 
for the project are nearly complete at the same time in 
the term as theproject was typically begun without these 
interventions. Almost all members of the team 
participate in the project substantially or are terminated 
by their team, in keeping with the terms of the team 
contract. After implementation of the contract and 
rewarded weekly meetings (Class D and E), teams 
actually terminated members, as happens in the “real 
world. Thus, two key problems have been addressed: 
procrastination and free loading . All groups agreed that 
their skill at working in groups had improved, but 
treatment groups D and E, where weekly meetings were 
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part of the grade, experienced strong agreement with this 
statement. 

Project Qu ality 

Professors judgedthat the quality ofthe final project was 
significantly improved as a result of incremental team 
developm ent. Using grades as an indicant of quality has 
proved difficult since gradestend to be relative to overall 
class perform ance. With incremental presentations, the 
expectation for the final project may shift as well. 
Students perceive that the quality of the project is 
improved as a result of team activity in all groups, but 
the strongest agreement is found in groups where team 
process interventions were employed, so student^ 
perception agrees with the professors’ perception on 
improved project quality. 

Satisfaction with the Team Experience 

Student satisfaction with the team pro cess imp roves with 
increased interventions and decreases with less structure 
or a lower level of interventions. The lowest satisfaction 
with the team project was experienced by Class B, a 
control team with the fewest interventions, and the next 
lowest, by Class G, students who had some interventions 
but who chose to not follow team process guidelines. 
The students with the highest level of interventions, 
Class D, experienced thehighest satisfaction (Appendix 
B). All teams using a contract strongly agreed with the 
statement in question 27, “The project team worked 
more effectively with the constraints concerning 
meetings and roles played than other teams on which I 
have worked which had no team process constraints. 
Teams with follow-up (whether rewarded or not) on 
regular weekly meetings strongly agreed with the 
statement, “I would choose the same team members 
again. 

When Students Choose 

The results with Classes F and G, where students chose 
whether to use team process interventions, are 
particularly interesting. While class B was a control 
class, students in the F group chose not to employ team 
process constraints. Highlights of these results follow. 

36% of the students chose to use the prescribed team 
process (intervention set F). They were not given any 
encouragement from the instructor. The guidelines 
were simply available to students as a link off the web 
page describing the team project. 



CS majors were evenly split between interventions F 
and G. IS majors were less likely to choose to use the 
team process (intervention F) only 24% of IS majors 
made this choice. 

Students with intervention set F w ere more like ly to be 
employed full or parttime, perhaps because they have 
a more mature understanding of the power of teams 
when run effectively. 

Students with intervention F were more likely to 
comment aboutincreased team effectiveness relatedto 
minutes and action lists, though minutes were required 
of all students. 

Students (F and G) recognize the effectiveness of 
regular, well-planned meetings, but are loath to take 
time outside of class. 

Several teams expressed frustration with students who 
contributed very little, procrastinated until the last 
minute, or simply had lower grade expectations. 

Many suggested that the derelict students be handled 
more formally, although an end-of-term peer 
evaluation accounted for 25% of the project grade. A 
few suggested the use of contracts. These comments 
originated with students involved in both interventions 
F and G. 

Students with intervention F more frequently 

expressedsatisfaction with the use ofemail and phone 
calls as a way to address differing schedules. 

Students with intervention G more frequently 

expressed a need for more detailed guidelines (!). 

Even students with extensive work experience express 
frustration about covering for the slackers in order to 
delivery a high quality project. 

One student expressed concern over the distribution of 
international students feltthathis/herteam contained 
too many international students which generated 
communication problems. 

One student expressed concern over the gender 
distribution of her team she was the only female. 

At the beginning of the term all students were asked 
for “team preferences in an on-line survey. The 
instructor attempted to use these requests to meet 
students’ perceived needs. At the end of the team 
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process, several students mentioned the importance of 
choosing their own teammates, most especially students 
in treatment G. Itis interesting to note that students with 
very specific ideas of who to work with were more likely 
to choose to follow treatment F (team guidelines). 

Also students who initially described the ideal team 
member using phrases like “achievers or “[students] 
who work hard were placed on teams together. At 
the end of the process, they were generally the most 
dissatisfied with the experience. Perhaps, students 
accustomed to star performers carrying the group, 
selected each other with this preference. The link 
between ideal team member definition and project 
outcomes bears further investigation. 

Students with some work experience are m ore likely to 
choose to follow a stmctured approach to teamwork. 
Therefore, it may be more important to required 
adherence to the team process when students have 
little personal experience with the world of work. 

Instructor 2 will experiment with contracts in the 
future. 

Instructor 2 will explore assigning student to teams 
based on work styles or habits, as with the Myers- 
Briggs instrument or similar. 

CONCLUSIONS AND FURTHER RESEARCH 

Our experience in the initial study involving Classes A 
and B, as well as this continuation with five more 
classes, leads us to believe that student team projects can 
teach effective teamwork and better prepare students for 
a team environment in the work world. Student teams 
can reinforce the learning of course content for all 
students, not just the stars who have been carrying the 
Joad for most student team projects. 

Two interventions appear to be critical. 

Weekly meetings as evidenced by meeting agendas, 
minutes, action lists submitted for grading purposes. 
Any absent student receives no points. 

Milestone presentations throughout the quarter timed 
to follow the topics currently being studied. Student 
teams receive feedback from peers and the professor. 



Our results are encouraging. Although much work 
remains to be done, we conclude that it is possible to 
dramatically increase learning, student team 
effectiveness, and both student and instructorsatisfaction 
with the experience when team process interventions are 
employed. A phased approach may be used (Stephens, 
2001). These interventions allow student teams to more 
accurately simulate the experience of professional 
business teams, thus better preparing our students for 
role on these teams in the work world. 
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APPENDIX A 
SURVEY 



Project Team Evaluation Name 

Part 1: Using the following scale, please evaluate the following statements. Please mark the accompanying scantron form. 
Please record your name on the scantron form as well as this form. 

A, Strongly Agree B. Agree C . Indifferent D. Disagree E. Strongly Disagree 

1. I like to work on a project team. 

2. I like to work alone. 

3. I prefer to do project work alone. 

4. I prefer to do project work as a member ofa team. 

5. Projects take more time if completed as a team activity rather than as an individual activity. 

6. Projects take less time if completed as a team activity rather than as an individual activity. 

7. Project quality is improved ifthe project is a team activity. 

8. Establishing a weekly meeting time helped the team to work on the project throughout the semester instead of as a “last 
minute endeavor. 

9. Publishing an agenda for each meeting was valuable. 

10. Publishing an agenda for each meeting caused us to use meeting time more effectively. 

1 1. Creating an action list was valuable. 

12. Creating an action list helped us to stay organized. 

13. Creating an action list helped us to make sure everyone did his or her “fair share. 

14. Recording and publishing the minutes ofmeetings was valuable. 

15. Recording and publishing the minutes of meetings improved communication. 

16. I learned how to use meeting time more effectively. 

1 7. My sk ill at workin g on team s has improved as a result of this experience. 

18. The project helped me to better understand topics and problems in the course. 

19. My participation on the project team will helped or will help me with the exam(s). 

20. Playing the role of facilitator was valuable to me. 

21. Playing the role of scribe was valuable to me. 

22. Playing the role of scheduler was valuable to me. 

23. Having a designated scheduler, scribe, and facilitator improved the team process. 

24. Having a designated scheduler improved the team process. 

25. Having a designated scribe improved the team process. 

26. Having a designated facilitator improved the team process. 

27. The project team w orked m ore effectively with the constraints concerning meetings and roles played than other teams on 
which I have worked which had no team process constraints. 

28. I would choose the same team members again. 
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APPENDIX B 
RESULTS FROM SURVEY 
AVERAGE, MODE, STANDARD DEVIATION 

(Please reference APPENDIX A for question numbers and TABLES 2 AND 3 for interventions) 



Set 


A 






B 






C 






D 






E 






F 






G 






Q# 


Avg 


M 


SD 


Avg 


M 


SD 


Avg 


M 


SD 


Avg 


M 


SD 


Avg 


M 


SD 


Avg 


M 


SD 


Avg 


M 


SD 


1 


3.76 


4 


0.94 


3.80 


4 


1.16 


3.92 


4 


0.91 


3.93 


4 


1.22 


4.16 


4 


0.91 


4.32 


5 


0.84 


3.83 


5 


1.18 


2 


3.33 


3 


0.8 


3.60 


3 


1.1 


3.68 


4 


0.99 


3.36 


4 


1.14 


3.44 


3 


0.99 


3.45 


4 


0.96 


3.43 


4 


1.07 


3 


2.67 


3 


0.91 


3.20 


5 


1.37 


3.08 


3 


1.12 


2.71 


3 


0.92 


2.76 


2 


1.17 


2.55 


3 


0.96 


2.86 


4 


1.24 


4 


3.86 


5 


1.01 


3.50 


4 


1.25 


3.88 


4 


0.88 


3.93 


4 


1.21 


3.96 


4 


0.93 


4.00 


5 


0.93 


3.86 


4 


0.97 


5 


3.05 


2 


1.36 


3.10 


2 


1.45 


3 


2 


1.22 


2.93 


2 


7.53 


2.84 


2 


1.34 


2.82 


2 


1.26 


3.14 


4 


1.35 


6 


3.38 


4 


1.36 


3.10 


5 


1.47 


3.6 


4 


0.96 


6 


4 


0.63 


3.40 


4 


1.33 


3.32 


4 


1.04 


3.31 


2 


1.32 


7 


4.1 


5 


1.02 


3.93 


4 


1.14 


4.28 


5 


0.74 


4.64 


5 


0.27 


4.29 


4 


0.61 


4.09 


4 


0.81 


3.97 


4 


0.95 


8 


4.43 


5 


0.68 








4.16 


5 


1.07 


4.93 


5 


0.73 


4.56 


5 


0.80 














9 


3.67 


4 


1.06 








3.2 


4 


1.26 


4.29 


5 


0.73 


3.82 


4 


0.87 














10 


3.76 


4 


1.09 








3.2 


4 


1.32 


4.07 


4 


0.65 


3.71 


4 


0.85 














11 


4.14 


4 


0.91 








3.52 


3 


1.16 


4.50 


5 


0.74 


4.02 


4 


0.68 














12 


4.29 


5 


0.85 








3.64 


4 


1.04 


4.36 


5 


0.77 


4.24 


4 


0.67 














13 


3.71 


3 


1.1 








3.52 


4 


1.19 


4.14 


4 


1.02 


3.73 


4 


0.90 














14 


3.62 


4 


1.2 








3.16 


4 


1.28 


3.57 


4 


1.07 


3.42 


4 


0.90 














15 


3.52 


5 


1.29 








3.08 


3 


1.22 


3.71 


4 


0.76 


3.49 


4 


0.93 














16 


3.9 


4 


0.94 


3.83 


4 


0.95 


3.88 


4 


0.78 


4.43 


5 


0.84 


4.11 


4 


0.76 


4 


4 


0.69 


4.03 


4 


0.73 


17 


3.9 


4 


1.09 


3.83 


4 


0.93 


3.64 


4 


0.95 


4.64 


5 


0.63 


4.49 


5 


0.62 


4.23 


4 


0.69 


3.91 


5 


0.98 


18 


4.38 


4 


0.59 


3.97 


4 


1.16 


3.72 


4 


1.14 


4.64 


5 


1.33 


4.56 


5 


0.77 


4.50 


5 


0.51 


4.47 


5 


0.61 


19 


3.33 


3 


1.02 


3.3 


3 


1.12 


3.36 


4 


1.25 


4.07 


5 


0.80 


4.20 


5 


1.05 


3.36 


4 


0.95 


3.11 


3 


1.05 


20 


3.52 


4 


0.98 








3.6 


3 


1.15 


4.21 


5 


0.78 


3.91 


4 


0.86 














21 


3.43 


3 


0.98 








3.32 


3 


1.07 


4 


4 


0.73 


3.87 


4 


0.97 














22 


3.29 


3 


0.9 








3.2 


3 


1.04 


3.93 


4 


0.97 


3.80 


4 


0.92 














23 


4.14 


4 


0.73 








3.4 


4 


1.32 


4.21 


5 


0.92 


4.04 


4 


1.04 














24 


3.52 


4 


0.93 








3.24 


3 


1.23 


3.93 


3 


0.89 


3.80 


4 


1.15 














25 


3.71 


4 


0.96 








3.28 


3 


1.28 


4.21 


5 


0.83 


3.84 


4 


1.08 














26 


3.95 


4 


0.74 








3.6 


4 


1.19 


4.29 


5 


0.65 


4.00 


4 


0.91 














27 


3.86 


4 


0.73 








3.44 


5 


1.39 


4.43 


5 


1.09 


4.29 


5 


0.78 














28 


4.19 


5 


1.17 


3.24 


5 


1.41 


4.4 


5 


1.15 


4.50 


5 


2.73 


4.33 


5 


1.03 


3.82 


4.00 


1.10 


3.69 


5 


1.32 
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THE MFC FRAMEWORK AS A CASE STUDY 
OF OBJECT-ORIENTED DESIGN 



Yonglei Tao 

Grand Valley State University 



INTRODUCTION 

In our “Graphical User Interface (GUI) Design” class, 
we use the MFC application framework (that is, the core 
of Visual C++) as a development tool. MFC not only is 
a powerful tool, but also offers a lot of educational 
benefits. Students, as they use the tool, learn design 
principles (such as “separation of concerns”), object- 
oriented application frameworks, common software 
architectures, and industrialcoding standards. However, 
considerable effort must be made to teach the 
framework, which often diverts much attention from the 
primary course objectives. 

W e use the MFC framework as a case study in the “GUI 
Design” class, which enables us to introduce features of 
MFC that students need to know in order to use it 
effectively, and also provides a context for teaching 
effective object-oriented design. Our experience shows 
that this approach gives students a unique learning 
experience while still allowing us to achieve our course 
objectives. 

OUR APPROACH 

Successful application frameworks are full of design 
patterns [1]. Examples of design patterns foundin MFC 
can serve as a basis forour lecture. UML [2] provides a 
notation to visualize object-oriented design techniques 
and makes it easy to deliver. 

We identify examples of design patterns in MFC and 
document them in UML (Unified Modeling Language). 
Then we organize and present our course material 
around basic design issues for GUI applications. 
Examples of design patterns found in MFC are used to 
facilitate our lecture. Although our primary 



consideration is software design, we make sure students 
will gain adequate knowledge aboutthe framework and 
be able to use it in a professional fashion. 

DESIGN ISSUES 

Our goals are twofold, teaching object-oriented design 
forGUI applications an d helping students understand the 
MFC framework. We emphasize on effective object- 
oriented design in the context of the MFC framework. 

We use several well-known design patterns [3, 4] to 
cover issues including Model-View Separation, 
Extensibility, Visibility between Objects, Navigability 
within Windows, and Persistent Objects. 

OBSERVATIONS 

We have taught the “GUI Design” class for four 
semesters. We treated the two course elements, object- 
oriented design and use of the tool, as separate ones in 
the first two semesters and then took the above- 
mentioned approach. Consistently giving a 
comprehensive project of the same type allows us to 
compare this approach with our earlier experience. 

We evaluate students’ projects according to the user 
interface design, functionality, and object-oriented 
structure. We found that 14 out of 21 students in the 
second semester had successfully completed their 
projects, 13 out of 18 in the third, and 15 out of 19 inthe 
fourth. A noticeable improvement was made due to this 
approach. Note that there was no comparable data in the 
first semester. 

Our approach helped students form a big picture about 
the MFC framework. As a result, students could handle 
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other details themselves, allowing us to stay focused on 
more important topics. Students still spent a lot of time 
searchingsolutions to certain design and implementation 
problems. We believe it is a useful experience for 
students. 
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THE TRIALS AND TRIBULATIONS 
OF UPDATING AN IS CURRICULUM 



Behrooz Seyed-Abbassi 
University of North Florida 



ABSTRACT 

The rapid and continual changes in computer technology necessitate the adaptation of curricular offerings in 
computer-related educational programs to provide students with the current skills and knowledge needed by today ’s 
organizations . The review and modifications involved in this process can present a challenging task for an academic 
program. At the University of North Florida , a group of faculty members , instructors , and advisors from the 
Department of Computer and Information Sciences conducted a comprehensive review of the Information Sciences 
and Systems (IS) Program to evaluate the program ’s structure, prerequisites, and course content. Over the period 
of a year, the review process and subsequent recommendations involved continual meetings, " spirited ” discussions, 
and persistent emails that resulted in curriculum changes which updated both the courses and the program structure. 
The integration of the changes into the IS Program continues to require the acceptance and support of 



administrators, teachers, students, and technicians. 



EXPERIENCES IN UPDATING 
AN IS CURRICULUM 

In the initial meetings, the often heated discussions to 
determine the preliminary groundwork brought with it 
the ominous realization that there would be many 
challenges in resolving issues that involved potential 
change. Some of the parameters, such as IS ’97 and 
ISCC’99, provided flexible guidelines for evaluation, 
while others, such as the Florida common course 
numbering system, comparisons of courses transferred 
from other educational institutions, and state educational 
requirements, caused more rigidity. Additional con- 
siderations, such as faculty strengths, domino effect on 
other courses, appropriate use of technology, and 
programming languages, that affected the strategies for 
revision were more subjective, eliciting a wide variety 
of ideas, concerns, and opinions as well as varying 
degrees of agreement or disagreement. The first course 
assessment resulted in an insidious loop that revolved 
around expansion of course content, state mandated 
course guidelines, and transfer students which took an 
inordinate amount of meeting time. It did prove to be a 
turning point in the discussions by changing the focus of 
the meetings more toward the structure of the program 



and general course content instead of the minuscule 
details of each individual course. This dramatically 
improved the productivity of the meetings and decreased 
the intensity of ownership issues on particular courses. 

The primary structural changes recommended were to 
replace Introduction to C with Introduction to Object 
Oriented Programming (OOP) and to modify the 
prerequisite courses to support a traditional path 
(introduction to programming and file structures) and an 
object oriented path (introduction to OOP and data 
structures using OOP) with both required prior to the 
database course. The group reviewed the content of 
each course and made adjustments to capture the 
necessary concepts as well as to ensure that the 
foundations were adequate for the succeeding courses. 
The approval of the recommendations did not mark the 
end of the revision process; it was the beginning ofmore 
changes that involved faculty, software, and student 
adjustments to facilitate the adoption of the 
modifications. The course content recommendations 
needed to be disseminated to affected faculty members, 
instructors, and adjuncts who then had to adapt to the 
new course and the changes in existing courses. 
Computer facilities needed to provide the hardware and 
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software to support the course requirements. Advisors 
guided students through written notifications, course 
adjustments, and curriculum counseling during the 
transition phase. Administrative support has been and 
continues to be essential for all areas affected by the 
program revisions and for the encouragement of further 
reviews. 

Overall, the responses to the curriculum changes have 
been positive from students and faculty. The addition of 



OOP has revitalized other courses by enabling new 
concepts and applications to be utilized. At the 
beginning of the meetings, some members expected 
wide sweeping changes and others hoped for a more 
conservative approach with the final result somewhere 
in the middle. The recommendations were aimed pre- 
paring the IS students with the foundations and skills 
required by prospective employers and businesses. The 
assessment process needs to be ongoing to evaluate the 
effectiveness of the revisions and to continue the process 
of essential changes for the enhancement of the IS 
Program. 
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THE EMPLOYMENT OUTLOOK FOR 
INFORMATION TECHNOLOGY WORKERS: 
A DEMAND FORECAST 

Rick Weible 
Marshall University 



EXTENDED ABSTRACT 



In the past few years demand for IT workers has 
experienced tremendous growth. This is the result of 
several factors, the major ones include Y2K and the 
eruption of the internet and dot corn’s. The current 
demise of dot corn’s may have eased the growing crisis, 
but it will return. On the supply side, the US has two 
basic sources for IT professionals: education or 
immigration. The US government controls immigration 
and state governments to a large extent control 
education. The shortage of IT workers may have a 
significant impact on the US and local economicactivity 
since most of the productivity growth experienced in the 
US in the past decade has been information technology 
driven. If the shortage of IT workers continues, it may 
have a stifling effect on economic growth. Much IT 
development is going offshore as local IT professionals 
are not available to do the work. Governmental policies 
may need to be changed to encourage more educational 
opportunities for potential IT workers. 

Three main questions need will be addressed. What is 
the size of the IT workforce? What is its growth rate? 
How does the lack of IT workers effect economic 
productivity? 

SIZE AND GROWTH RATEOF IT WORKFORCE 

There are several sources and estimates of the number of 
IT workers. The US department of Labor says there are 
2,347,030 Computer Occupations in the US in 1999 and 



projects the number will grow to 3,050,000 by the year 
2008. General employment number by major IT 
manufacturers can be used as a surrogate. Chart 1 
shows the monthly growth in Computer and Data 
Processing Services Workers between 1972 and 2001. 
This is base on SIC = 737. 

Chart 2 shows the growth in the average weekly 
earnings for these workers during the same period. 

Regardless of the source there is clearly an increase in 
the number of IT workers over the past 30 years. With 
a dramatic acceleration in the growth rate starting in 
1995. Two factors have been driving this growth, the 
Internet and Y2K. With Y2K behind us and a slow 
down in the growth of the Internet, will we continue to 
see this high rate of growth in IT workers? The US 
Department of Labor predicts a continued high growth 
rate. 

HOW DOES THE LACK OF IT WORKERS 
EFFECT ECONOMIC PRODUCTIVITY? 

This growth rate has lead to a shortage in IT workers. 
What is the effect of this shortage on the economy? 
Many economists have contributed the economic growth 
of the 1990’s to productivity gains from the use of IT. 
If this is true, then the lack of IT workers should have a 
stifling effect on the economy. We currently are seeing 
a slow down in the economy, is this the cause? 
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CHART 1 

COMPUTER AND DATA PROCESSING 
SERVICES WORKERS BETWEEN 
1972 AND 2001 






ieMco 










s 


























tJt 





S' s s s s s s s s s s s s / s 
CHART 2 

AVERAGE WEEKLY EARNINGS 




HOW DOES THE LACK OF IT WORKERS 
EFFECT ECONOMIC PRODUCTIVITY? 

This growth rate has lead to a shortage in IT workers. 
What is the effect of this shortage on the economy? 
Many economists have contributed the economic growth 
of the 1990’s to productivity gains from the use of IT. 
If this is true, then the lack of IT workers should have a 
stifling effect on the economy. We currently are seeing 
a slow down in the economy, is this the caused? 
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Extended Abstract 



This contribution deals with an approach to teaching 
information technology (IT) and system architectures 
that integrate the theory and practice using a real-world 
case study. The rationale of the approach is that an IT 
architecture should be designed in terms of an 
architecture framework that relates the organizational 
and IT resources, thereby enhancing the organization’s 
strategic decision-making capability and ability to 
leverage the organization’s competitive position. 
Business drivers and initiatives are translated to IT 
requirements and guidelines for the acquisition and 
deployment of IT’s. 

The concepts of systems architecting are applied to a 
case study undertaken in team mode, with the teams 
modeling the application, data and physical 
infrastructure perspectives, respectively. The case study 
described new enhancements to an E-Business System 
that each team had to analyze and model within the 
context of their viewpoint. 

The architecting framework that is used is the Index 
Model. The architecting process is a staged model that 



allows for iteration and incremental enhancement of the 
models. The architecting methodology, based on the 
architecting process model, was defined in terms of a 
series of steps that are followed by each of the project 
teams. Modeling was performed using appropriate 
notations to design the systems at various levels of 
abstraction. The resultingIT and system architectures are 
documented by the project teams following the EAB 
architecting documentation method, and the deliverables 
are presented to the class and project sponsors on 
completion of the project. Various industry standards are 
relevant to the syllabus for such a course. 

Following upon teaching the course the outcomes have 
been recorded in terms of the course goal and objectives. 
The approach was followed for a three credit hour 
elective in a Master of Science in Computer and 
Information Systems (Software Management) Program 
offered in the College ofBusiness Administration at the 
University of Detroit Mercy. 
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ABSTRACT 

MBA candidates need to understand the connection between an increased use of the Web by companies and 
intellectual property rights . Business students must be ever mineful of these rights given the precarious legal 
implications inherent in Internet-driven enterprise. Corporations , as well as writers , scholars and artists , want to 
reap economic benefits from the intellectual property that they create , whether in real space or cyberspace. There 
is an increased willingness of the nation’s judiciary to use the constraints of intellectual property law to send a 
message to those who are using emerging technologies that intellectual property rights are alive and well in the 
digital era. This paper focuses on intellectual property policy and its effect on business modalities in light of the 
explosion of the Web. Intellectual property rights such as copyright, trademark and patent are defined and discussed 
with an emphasis on what the business manager (and the aspiring business manager) need to know before adopting 
new technologies or implementing proprietary systems. 



INTRODUCTION 

The legal landscape with regard to the Internet is a vast, 
uncharted territory which has been likened to the Wild 
West (Gruley and Simpson, 2000). While many are 
wrestling with this brave new on-line world, lawmakers 
have been reluctant to regulate the Web, opting instead 
to allow Internet technology to develop unhampered. In 
this laissez faire environment, it is argued that creativity 
may flourish and society as a whole may become the 
ultimate beneficiary (Bessen and Maksin, 1997). Such 
reticence to act may be predicated upon a willingness to 
promote the free flow of information that has been the 
hallmark of the Internet. 



Businesses are struggling to cope with the fast-paced 
evolution of technology where the law is not yet fully 
developed to deal with such issues as privacy, taxation 
and the use of copyright laws in the digital era. Indeed, 
the Internet has fast pervaded every facet of life from 
communications with loved ones to business-to-business 
transactions in far-flung locales. Traditional laws are 
being applied in new ways to deal with these issues of 
first impression. Legal questions involving e-mail in the 
workplace, Web content, and domain name protection 
are finding their way into our nation’s courts with little 
or no legal precedent. 
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As information systems evolve and new technologies 
change the way we do business on a daily basis, the 
business manager must be wary of the possible 
infringement of intellectual property rights as he or she 
uses and disseminates information. This idea of 
emerging technologies as property that is owned and 
protected by law must be taught to the MBA candidate 
to foster a respect for the protection of copyrights, which 
is a basic facet of free enterprise and creative society. 
Students must be made aware that newly evolving 
technologies are copyright, trademark, or patent-worthy 
and thus carry significant legal ramifications requiring 
licensing. 

INTELLECTUAL PROPERTY 

Intellectual property policy is based on the desire to give 
incentives to authors and inventors to create new ideas 
as balanced against the desire for society to benefit from 
the dissemination of those ideas (Bessen and Maskin, 
1 997). Intellectual property is an intangible creation that 
is legally protected, just as one’s ownership of real 
(land) or personal (i.e., car) property is protected by the 
law (Sartori, 2000). Intellectual property represents the 
creative ideas that give businesses a competitive 
advantage (Bush, 2000). Although the Internet makes it 
very easy to copy information, businesses that have 
invested great resources want to protect their 
investment. There are actually four types of intellectual 
property: copyrights, trademarks, patents, and trade 
secrets. 

COPYRIGHTS 

Copyright law is derived from the United States 
Constitution (Article I, Section 8, Clause 8). The Copy- 
right Act of 1976, which is the current copyright law in 
this country, protects original and creative works that 
are fixed in a tangible medium of expression. In short, 
copyright protects creative expression such as literary 
works, musical works, dramatic works, pantomimes and 
choreographic works, pictorial, graphic and sculptural 
works, motion pictures and other audiovisual works, 
sound recordings and architectural works (Sartori, 
2000). Copyright does not protect facts, ideas, 
procedures and methods which are covered under 
patents, nor does it cover titles, names or slogans which 
can be afforded trademark protection. 

Authors and inventors primarily benefit from a 
registered copyright. Copyright protection begins at the 
moment the original work is created and “fixed” in a 



perceptible medium, and endures for a period of the life 
of the author plus seventy years. A copyright affords 
the owner a bundle of valuable and exclusive rights 
including the right to copy the work and the right to 
distribute it. Any interference with these rights, such as 
the unauthorized or without consent use of the 
copyrighted expression, constitutes infringement for 
which the law imposes substantial penalties. The 
Internet has encouraged the desire of individuals for 
content such as text, images and music. This in turn has 
increased copyright violations (Joss, 2000). Content 
companies would like copyright laws to be strengthened 
so that they have greater control over electronic 
distribution (Bowman and Borland, 2001). 

TRADEMARKS 

A trademark, by contrast, is defined as a word, name, 
symbol or device, or combination of these that serves to 
identify the source of products or services and to 
distinguish then from the products or services of others. 
Therefore, a trademark is a brand name which is used to 
distinguish one company’s product from that of a 
competitor (Saunders, 2000). Unlike the duration of 
copyright, trademark protection only lasts for as long as 
it is in actual use. Typical trademarks include word 
marks (Bayer), symbols, (Nike swoosh), numbers 
(Boeing “747”), colors (Owens Coming “pink” 
insulation), and sounds (NBC chime). To enforce a 
trademark, one must (1) prove ownership of a valid 
trademark, and (2) prove that the user’s actions are 
likely to cause confusion and/or dilute the strength of 
the brand name (Saunders, 2000). 

Trademark issues arise on the Internet when unautho- 
rized references to existing trademarics are made in 
domain names. So called “cybersquatters” commit a 
form of cyberpiracy when, in bad faith, they seek to “use 
an Internet domain name that is identical or confusingly 
similar to the trademark,” (Gollin, 2000) or is “dilutive 
of another’s famous trademark” (Satterthwaite, 2000). 
On November 29, 1999, President Clinton signed the 
Intellectual Property and Communications Omnibus 
Reform Act of 1 999 which allows a trademark owner to 
seek relief under the section entitled “Anticyber- 
squatting Consumer Protection Act.” One example of 
the magnitude of this problem is that of the Porsche 
brand of products. At any one point in time there are 
over 100 unauthorized references to the trademark 
“Porsche” in domain names. Recently, the International 
Olympic Committee dealt with the cybersquatting 
problem (Barringer, 2000). Two months before the 
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Games began in Sydney, the IOC filed a lawsuit against 
1800 cybersquatters who had registered domain names 
using terms owned by the IOC. The task of defending a 
brand name to prevent its dilution is a daunting one 
indeed. To protect its intellectual property rights, the 
IOC hired 20 private Web monitors on three continents 
to monitor possible violations. 

PATENT PROTECTION 

While copyright protects “creative expression” and 
trademark protects “brand names/symbols,” a patent 
protects the underlying “ideas” and “inventions.” As 
defined by Black’s Law Dictionary, a patent is a “grant 
of a right to exclude others from the making, using or 
selling of an invention during a specified time; it 
constitutes a legitimate monopoly.” A U.S. patent does 
not provide the right to practice an invention, Rather it 
provides the right to exclude others from making, using, 
selling, etc. one’s invention. This exclusive right 
endures for twenty years from the filing of the patent 
with the Patent Office. 

Recent changes in the Patent Office Guidelines and in 
case law have paved the way for an opening of the 
floodgates for the patenting of computer and software 
inventions. In 1996, the U.S. Patent and Trademark 
Office issued its “Guidelines for Computer- 
Implemented Inventions” which made e-commerce, 
Internet and business inventions eligible subject matter 
for U.S. patents (Sartori, 2000). Prior to this time the 
patent office was not receptive to the patenting of 
computer-related inventions. Indeed, e-commerce 
patents are “one of the fastest growing arenas for patent 
protection in the United States” (Gollin, 2000). 

A recent Federal Circuit Court decision further extends 
patent protection in the Internet field. In State Street 
Bank & Trust Co. v. Signature Financial Group Inc., 
(149 F.3d 1368 (Fed. Cir. 1998)), the Federal Circuit 
reaffirmed the ability to patent computer-related 
inventions and first announced that business methods 
are suitable patent material. As such, a patent may be 
obtained for an invention in e-commerce like a new 
method of marketing or distribution over the Internet. 
This is extremely important for the business community 
as new ways of doing business in the digital world are 
being created daily, like UPS’ package tracking 
technology which has redefined the package delivery 
business (Laudon and Laudon, 2000). Business 
managers should take note of the intellectual property 
rights that may be available to the business enterprise 



and should be wary of infringement activity (Sartori, 

2000 ). 

RECENT LEGISLATION 

Fueled by the new-found politicism of some of the dot- 
com companies, new legislation has emerged to deal 
with the volatile, digital electronic environment (Gruley 
and Simpson, 2000). The Digital Millennium Copyright 
Act [hereinafter “DMCA”] was passed in 1998 and 
provides strong copyright protections to deal with the 
issue of infringement in cyberspace. “The DMCA was 
intended by Congress to facilitate the robust 
development and world-wide expansion of electronic 
commerce and communications over the Internet.” 
(Senate Report no. 190, 105th Congress, 2nd Session, 
1998). The speedy passage of the DMCA was seen as a 
joint affirmation by Congress and the President of the 
importance of copyright law in the electronic 
marketplace. 

An important provision of the DMCA is the “Safe 
Harbor” provision which serves to protect Internet 
service providers and limit their liability for the mere 
transmission of material through its system or network. 
It is arguable then that Congress intended with this 
provision to shield ISP’s from the bulk of copyright 
compliance and thus place the obligation of compliance 
on the end user of the service. 

Another provision of the DMCA that has direct 
application to the Internet is that of “Fair Use.” This is 
a defense to a finding of copyright infringement that 
asserts that the use is fair and acceptable. It is loosely 
based on the Constitutional right of free speech as 
protected by the First Amendment. To judge whether 
the use is “fair” within the marketplace of ideas, courts 
will look at four factors as follows: (1) the purpose and 
character of the use (i.e. commercial, non-profit); (2) the 
nature of the copyrighted work (factual works get less 
protection than creative works); (3) the amount taken in 
relation to the whole; and, (4) the effect of the use on the 
potential market for the copyrighted work. 

The European Union is attempting to draft similar 
legislation to the DCMA. The European Copyright 
Directive hopes to establish the rights of individuals 
who want to create, produce, sell, transmit or use 
something protected by copyright in a digital 
environment (Mitehener, 2001). 

CONCLUSIONS 
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Companies need to be aware of the demands of existing 
law. Indeed, “many businesses do not even realize that 
they are creating or using copyrighted works on a 
regular basis.” (Saunders, 2000). The business manager 
should also be made aware of changes to existing law 
that affect their information systems or licensed 
properties. New legislation such as the DMCA should 
not be ignored. Tremendous opportunities and great 
risks abound. 

Arguments can be made for both positions; for increased 
regulation in the Internet industry and for unhampered 
freedom of expression. Certainly the United States 
Constitution protects the concerns of each side. The law 
has yet to fully develop in the area of the Internet. 
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Extended Abstract 



In dynamic computing affiliated disciplines it is critical 
for schools to periodically re-evaluate their curricular 
offerings in order to produces graduates who possess 
relevant knowledge, skills, and abilities (KSA). As an 
applied discipline information systems programs needto 
align their curricular offerings with the current and 
future needs of industry. To this purpose, university IS 
programs seek ongoing curricular guidance from 
industry constituencies, from professional organizations 
such as the Association of Computing Machinery 
(ACM), the Association of Information Technology 
Professionals, and others. These effo rts have resulted in 
curricular guidelines such as IS’97 (Davis, 1997) and 
ISCC’99 (Lidtke, 1999), and has also served as input 
toward pending accreditation efforts (Gorgone 99). 

While generalized curricular models provide welcome 
guidance in specifying which knowledge, skills, and 
abilities (KSA) are generally important, it is incumbent 
upon individual programs to adaptthese recommendations 
based on regional needs and changing State-of-the-Art and 
State-of-the-Practice methods. 

Rapid change within the computing industry and in the 
software development arena suggests IS programs 
should assess which development methods, techniques, 
and tools are practiced currently and within the 
foreseeable future. Recent re search (T ang, 2000 ) reports 
system development methods to rank high in requisite 
proficiency level, and using empirical support suggests 



student achievement is deficient regarding knowledge of 
software development methods. Yet the referenced 
research used a macro level of granularity and provides 
no elaboration concerning which development methods 
are important and which have insufficient student 
achievement. Similarly, industry publications (such as 
Computerworld, 2000) regularly report on hot pro- 
gramming skills and tools yet they rarely specify areas 
comprising important development methods and 
modelingtechniques. Importantly, shifting development 
paradigms, such as from structured to object oriented 
methods, requires substantial curricular change in the 
emphasized KSAs related to development methods and 
modeling techniques. Whereas anecdotal evidence, a 
desire to be “leading edge,” and input from particular 
industry partners has convinced some programs to 
transition from structured to object oriented foci, other 
programs are seeking empirical evidence to support their 
decisions regarding which modeling techniques to 
emphasize. Knowing which techniques are currently 
used (state-of-the-practice) and will likely be used in the 
future (state-of-the-art) would enable programs to make 
more informed decisions regarding paradigm transitions 
and which techniques to emphasize within their chosen 
development paradigm. 

Although there is an abundance of anecdotal claims 
suggesting movement toward object oriented approaches 
and the emergent Unified Modeling Language (UML) 
modeling notation, there is a dearth of empirical 
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evidence regarding which analysis and design methods 
and modeling techniques are currently applied to 
software development projects. To date, empirical 
research concerning state-of-the-practice development 
techniques does not suggest industry adoption of the 
object oriented developm ent paradigm (G lass, 1 999). In 
addition to current practices it is important to also 
investigate anticipated transitions, thus preparing 
students for which techniques will be applied to real- 
world modeling projects within the foreseeable future. 
Such empirical evidence is needed if faculty are to make 
informed decisions regarding which techniques to 
emphasize in IS curricula. 

The purpose of the current study is to empirically assess 
the extent to which particular development methodo- 
logies (SASD vs. 00), techniques and tools are currently 
practiced and to assess practitioner perceptions of 
method usefulness and their expectations regarding 
future use. 

The target sample for the study is project managers. 
Questionnaire based surveys are being conducted to 
assess degree of methodology, technique, and tool usage 
and perceived usefulnesson their current and foreseeable 
projects. 

Descriptive statistics will be employed to rank perceived 
usage and usefulness of various modeling tools (e.g., 
ERwin vs. Rational Rose) and techniques (e.g., Entity 
Relationship Diagram vs. Class Diagram) and to forecast 
anticipated changes in usage patterns. 
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Extended Abstract 



In today’s technology-based industry the need to be 
competitive is based on how well and quickly the 
corporationand its employees respond to the adoption of 
new applications, different business processes, new 
hardware and different development methodologies. In 
a similar manner, universities are being challenged to 
react with a new paradigm generated by the rapidly 
emerging technologies. Investigating corporate stra- 
tegies may provide some guidance to those teaching in 
a university environment. 

Success depends upon how well corporations provide 
their employees with the resources necessary to 
effectively use these technologies, thereby enabling them 
to deliver new products or services to the clients. This 
is in many ways a paradigm shift because it requires a 
change in cognitive process concerningthe approach to 
a certain business process or a design of applications 
while in the process ofadopting new learning processes. 
It can be assumed that the employees are also 
contributing to the knowledge base repository in the 
corporations, which is fundamental to the technology 
driven industry. Additionally, there has to be aseamless 
alignment among the pre-existing corporate environment 
since any misalignment may not only cause tremendous 
burden on the corporation and its clients, but 
economically it may be disastrous. To overcome these 
issues the corporations are aggressively adopting 



industry wide in-house educational and knowledge 
enhancement programs. While these programs may not 
be in direct com petition with a university curriculum, the 
university is often pressured to teach similar skills. 
Examining the issues faced by corporations may help 
universities address these demands, especially to 
differentiate between education and training. These 
corporate programs may also have some shortcomings. 

Knowledge management and e-learning methodologies 
are driving change in the instructional environment. 
Because they both require the same infrastructure, these 
two new methods of acquiring knowledge can befurther 
broken down into self-paced or on-line interactive 
learning. These two methods create a knowledge base, 
which creates a learning platform, which provides the 
need-based instruction to individuals. 

The questions that this paper addresses are: 

• Are these new methods actually benefitting the 
technology driven economies and are the user/ 
consultants/developers accepting this paradigm? 

• Can universities apply some of these new learning 
environments to either improve current curricula or 
create new ones? 
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This research studies a set of corporations and the 
processes they have adopted in their corporate wide 
employee knowledge enhancement programs. Some- 
times the new process works and sometimes it does not; 
this study identifies both. This study also examines the 



cost benefit of e-learning and e-training. It also looks 
into an employee's comfort level with different 
instructional environments. It is anticipated that other 
issues will be discovered in this researc h that will benefit 
both corporations and universities. 
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ABSTRACT 

Collaborative learning could be defined as a situation in which two or more peoplelearn or attempt to learn together. 
In such a situa tion, students could ben efit from working together, but there are also a lot of difficulties to overcome 
in this environment. The Department of Informatics at the University of Pretoria challenge their third year students 
with a major project where they have to develop an Inform ation system for a real clien t. While working on the project, 
the students are implicitly exposed to communication and interpersonal skills, as defined in the IS’ 97 curriculum. 
Communication skills include listening skills, negotiation skills, interviewing skills, facilitation skills, observation 
skills and presentation skills. The skills of leadership, small group communication skills, small group organization 
and working with diversepeople are listed among the interpersonal skills. In the groups the students are confronted 
with a lot of problematic situations, such as conflict between the members, and dominant group m embers. The article 
discuss these problems under the following headings: One group member does all the work; Dominant group 
members; Self-oriented need of group members; Logistical problems; Inability of a group to come to a decision; 
Cultural diversity and Groups being more socialistic than productive . The advantages of collaborative learning are 
evaluated based on a classical case study, Twelve Angry Men, where four important factors regarding teamwork, and 
team dynamics are discussed. These factors are briefly discussed as Leaders Engage Inquiry; Vigorous Pursuits of 
Decisions; Conflict Serves the Group Process and the Benefit of Varying Backgrounds. All of this is then applied and 
evaluated based on the actual projects of the third year students, and the practical findings are discussed in detail 
under above-mentioned headings. Finally, a brief preview is given on future work regarding the teams and the 
projects in the final year of study for the Information Systems students. 



INTRODUCTION 

With the rapidchangingenvironmentandthecontinuous 
pressure on the Information Systems environment in 
South Africa, as well as in the rest of the world, the 
challenge lies in thehands ofthe lecturers of Information 
systems students to make sure that they are trained and 
equipped for the real world. The University of Pretoria 
is currently focusing all of their education to be 
internationally competitive, but locally relevant. When 
looking at the IT environment, this means that the 



students must be equipped with both the theory of 
Information systems and the skills of analysis, modeling 
and programming. All of this could be taught in a 
classroom situation, and practical sessions, but some of 
the skills required by the IS’97 curriculum for IS 
personnel in the modern world, are not addressed in a 
classroom situation . As mentioned by, Thomas (2000) in 
her Doctoral theses on group work, most of these skills 
are implicitly enforced on the students by placing them 
into groups. In the Department of Informatics the 
students are challenged with a major group project in 
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their final year. The focus of this paper is to look into the 
group work of these students. 

This paper is set up as follows. The teamwork and colla- 
borative learning section will discuss some theoretical 
issues concerning teamwork and collaborative learning. 
The projects in the third year section will describe the 
third year project by looking at the group work situation. 
Before the conclusion, the expectations and future work 
section will briefly list our expectations and future work 
with these projects. 

TEAMWORK AND 
COLLABORATIVE LEARNING 

According to Pierre Dillenb ourg’s book, Collaborative 
Learning — Cognitive and Computational Approaches , 
collaborative learning could be defined as a situation in 
which two or more people learn or attempt to learn 
together [Dillenbourg, 1999]. In trying to attempt this 
goal of learning something, there are several factors that 
work into a group or a team of people. In a classical case 
study, Twelve Angry Men [Twelve Angry Men, CASE 
STUDY], four important factors regarding teamwork, 
and team dynamics are discussed. These factors could be 
of great value to the studentin the learning environment 
if they are applied correctly. These factors would now 
briefly be discussed. 

Leaders Engage Inquiry 

Effective team leaders en courage e veryone to participate 
on an equal basis. The leader does not impose his or her 
views, but uses open inquiry to weigh the facts and 
clarify ambiguities. It takes courage for a leader to 
operate on equal terms with other people and to resist 
taking over the discussion or pulling rank. 

Vigorous Pursuits of Decisions 

A work group entrusted with a decision cannot allow 
itself to avoid its responsibility, but must stay together 
and work thro ugh its differences. As discussions heatup 
and becom e comp licated, team s often look for a way out. 
They try to find a “quick fix of limit their discussion to 
a part of the problem in order to escape from a messy 
situation. Although teams are usually not locked up in 
conference rooms like a jury, they need to understand 
that they’re accountable for making a good decision. 



Conflict Serves the Group Process 

When team members have a common goal and feel 
accountable to each other, conflict can be safe ly used to 
understand varying points of view. When there is trust 
among team members, conflict can generate new ideas 
and breakthroughs. Conflict highlights different points 
of view and the way each person sees things. Without 
trust, however, conflict becomes destructive and 
damages the team. 

Benefit of Varying Backgrounds 

Quality decisions depend on tapping the diversity of 
experience present in the group. The term “diversity is 
often limited to differences of color, nationality or 
gender, but the true value of diversity is the wealth of 
experience and perceptions brought by each individual 
to the team. Teams learn that different people because of 
their unique life experiences see information or ‘facts’ 
differently. Decisions are most effective when a diversity 
of viewpoints is available to the team. 

When looking at the IS profession, there are a great 
emphasis on teamwork and group working skills. These 
skills are also recognized by the curricula for IS students. 
The IS’97 Curriculum [Davis et al., 1997] recognizes 
this need for the softer skills and has communication 
skills and interpersonal skills as two of the main 
characteristics needed by an IS graduate. This is shown 
inTable 1 . Communication skills include listening skills, 
negotiation skills, interviewing skills, facilitation skills, 
observation skills and presentation skills. The skills of 
leadership, small group communication skills, small 
group organizatio n and wo rking with diverse pe ople are 
listed among the interpersonal skills. 

Personal and interpersonal skills are also deemed 
important in the Informatics Curriculum Framework 
2000 (ICF-2000) for Higher Education [Mulder & van 
Weert, 2000]. In this, they’ve identified three categories 
of Information systems professionals, namely 
informationusers, information appliers and information 
workers. Out of this, the ICF-2000 even suggests that 
communication skills, and interpersonal skills should not 
only be part of the curriculum, but should be credit 
bearing. 
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TABLE 1 

CAPABILITIES AND KNOWLEDGE EXPECTED FOR IS PROGRAM PROFESSIONALS 

[Davis et al., 1997] 



Characteristics 


With the ability to... 


Using the knowledge of... 


Communication 


• accurately observe, note and explain 
observations of events; 

• actively listen and express complex 


• listening, observing and documenting; 

• interviewing and speaking; 




ideas in simple terminology; 


• negotiation and facilitation; 




• organize and make presentations; 


• presentation and interpretation of data; 




• write memos, reports and 
documentation. 


• multimedia development and utilization; 

• computer and video conferencing 
techniques. 


Interpersonal 


• effective work with people of diverse 


• leadership, management and 


Relationships 


backgrounds; 


organizations; 




• effectively w ork with people at all 
corporate levels; 


• small group communication and 
motivation; 




• lead and facilitate teams in a 
collaborative environment; 


• organization, team and individual goal 
setting; 




• develop win-win approaches; 


• share vision and responsibility; 




• empathetically listen and seek 
synergistic solutions. 


• cultural diversity. 



To conclude this discussion on teamwork and colla- 
borative learning, we would briefly look into some 
problems expected with group work. Research has 
shown that there are many advantages to working in 
groups, but there are also several problems. Some of 
these problems will now be discussed. 

One group member does all the work. Some people 
are perceived to do most of the work while other 
students depend on the good students in the group. The 
less able team member leaves it to the others to do all the 
work, and thus creating a “free riding effect. The more 
able team member, who is doing all the work, feels like 
a “sucker . This situation is also referred to as “social 
loafing [Abrami et al., 1995]. One of the suggested 
solutions to this problem is that the participant must 
change from listener, observer and note taker to active 
problem solver, contributor and discussant [Daigle, 
1996]. 



Dominant group members. Some students are merely 
shy and do not participate for this reason. Some students 
feel that being put in the public eye is a position of risk 
[Thomas, 2000]. Others might dominate the group, even 
if the others would like to contribute, those students take 
over. The dominant student is often egoisticand refuses 
to consider the perspectives of others in the group. 

Self-oriented need of group members. Some students 
have self-oriented needs and this can interfere with the 
work ofthe group as a whole. Forinstance, students with 
high-level abilities complain about being held back by 
slower team members. Students then often decide to 
divide the work; one might be the thinker, while the 
other is the typist. Another situation might be when one 
group member does all the work for one assignment, or 
deliverable, while the other student do all the work for 
the next [Thomas, 2000]. 
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Logistical problems. There can be problemsif students 
are expected to do group work outside of class time. A 
great deal of time can be spent trying to get everyone 
togetherfor meetings, especially if some students are not 
motivated. 

Other problems such as inability of a group to cometo 
a decision, cultural diversity problems based on 
knowledge and trust, groups being more socialistic 
than productive are also discussed in a thesis by T.A. 
Thomas, “A Teaching Environment for Learning Soft 
Skills Applicable to Information SystemsDevelopment 
[Thomas, 2000], 

The next sectionwill explain the group assignments and 
project the third year students have to complete as part 
of their final year of undergraduate studies in 
Information systems at the University of Pretoria. 

PROJECTS IN THE THIRD YEAR 

The purpose of this course is to give students the 
opportunity to be part of an information development 
project that is as near to real-life as is possible. They are 
expected to analyze, design, test and implement an 
information system, for a real user with a real 
information system need, as part of aproject team. The 
course aims to establish and integrate the theoreticaland 
practical work done in the first two years of study and to 
prepare students better for the working life. 

Students must work in a project team, consisting ofthree 
or four members as chosen by themselves. The 
teamwork teaches students to work together in groupson 
a very important project (their degrees depend on it) and 
to learn how to handle the inevitable conflictthat arises 
out of it. The teams operate without aproject leader and 
have to resolve leadership roles among themselves. 

Although specific work is allocated to individual team 
members, all team mem bers must understan d all aspects 
of the system at all stages of the development. This 
includes both functional and technical issues. This is 
tested during every deliverable of the system. For 
instance, although one team member will probably 
research how to setup and use the bar code reader, all 
team members must be able to do it as well. 



The benefits of the project can only be achieved if the 
project is big enough and complex enough. See 
Appendix A for a description of the quantity and size of 
the project. 

A fairly standard System Development Life Cycle 
(SDLC) is followed and students have the choice of 
using traditional (data-flow diagrams, ERDs) or object- 
oriented techniques (UML). They have to produce the 
following deliverables and demonstrations. 



TABLE 2 

THE PROJECT DELIVERABLES 



Delivera ble 


Weight 


Deliverable 1 Project proposal 


1.0 


Deliverable 2 Analysis, 

feasibility study and detailed 
requirements 


2.0 


Deliverable 3 Functional 
specification 


2.0 


Deliverable 4 Technical design 
and test specifications 


2.0 


Deliverable 5 Maintenance 
document, user manual and 
training material 


1.0 


Demonstration 1: Technical 
features demonstration 


2.0 


Demonstration 2: Working 
prototype 


1.0 


Demonstration 3: Final 

demonstration and project day 


3.0 


TOTAL 


14.0 



Deliverables are marked during a meeting with the 
course coordinator of between 30 minutes and one hour 
depending on the type of deliverable. Certain 
demonstrations are also given where students are 
expected to do a formal presentation. Some of these 
demonstrations constitute the examinations and are 
marked by between two and three lecturers, depending 
on the type of demonstration. These sessions teach 
students how to present completed work to their users, 
management and other interested parties. 
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Other courses in the third year are supplementary to the 
project work done. An advanced Delphi and network 
course is given in the first term to prepare students. In 
the second term (when conflict and tension is normally 
at its highest) a course is given that considers, among 
other things, teamwork and project management. 

Practical Findings 

• Students mostly exceeded our expectations. They work 
harder than anticipated and the course coordinator in 
many cases had to advice them to work less. One of 
the big problems is ensuring that students don’t 
neglect their other subjects. At certain points during 
the project students can work up to 60 hours per week 
each. 

• Most of the team problems and advantages discussed 
in Section 2 were experienced by the teams. The most 
common problems were: 

• One group member does all the work. Although 
students choose theirown teams, quite a few of the 
teams build up a lot of interpersonal tension. The 
biggest source of tension is one or more members 
not doing their part. In most cases these issues can 
be resolved in conjunction with the course 
coordinator. In severe cases were communication 
have broken down completely, team members 
communicate formally by letter with the course 
coordinator as middleman, 

• Self-oriented need of group members. Another 
source of tension in the groups are the different 
levels of commitment by each team member. Some 
team member’s commitment is towards a winning 
project, while other team members barely want to 
complete the project in order to obtain the credits for 
the course. 

• Conflict serves the group process. In most cases, 
conflict in the group is because of different 
perspectives on the solution. This conflict often 
leads to a better solution of the problem at hand. 
Only in very extreme cases, the conflict leads to 
total breakdown of the groups, or even the inability 
to complete the project. 

• Logistical problems. For the bigger teams, 
logisticalproblems are often theirdownfall. Much of 
their time often go into a process of finding a time 
that will suit everyone, and even when finding a 



suitable time, it could be at an impossible time of the 
day and night, or even with a lot of interruptions. 
This also disables the groups to put in long 
productive sessions for effective group work. 

EXPECTATIONS AND FUTURE WORK 

• The ultimate goal to work for isto prepare the students 
for the real world as good as possible. This can only be 
done by enhancing the pressure and diversity of the 
group, and to focus more on the interpersonal and 
communicational skills of the students. 

• Out of the experiences from some of the groups, 
especially the groups that failed to succeed, some 
extremely valuable lessons were learned. Since the 
main reasons for failure were either inter-team 
problems, or the inability for estimating the sizeof the 
project, and thus starting too late, project managers 
will be appointed in the future. The role of this project 
manager would be to, firstly, resolve problem issues 
inside the groups, and secondly to plan the project 
throughout the SDLC. 

• Most of the students choose their teams themselves; 
therefore the groups are extremely homogeneous, in 
terms of language, culture and even backgrounds. As 
mentioned in Section 2, diversity can benefit thegroup 
work and contribute to a better understanding of the 
problem and even a better solution to the problem. 
Therefore, combining diverse people into one group 
can be considered in the future. 

CONCLUSION 

Supplementingformal learning with collaborative group 
learning ensures a number of benefits. 

Students learn some of the important aspects that can’t 
really be taught in a formal class situation. Most students 
still have a very mechanistic view of developing 
systems. In this view problems are straightforward and 
can be resolved if you just use the right technique. Even 
after formal lectures on the softer issues involved with 
systems development students still have a very naive 
view of what systems development is all about. When 
put into teams, they find, by first hand experience, that 
the softer issues are extremely important. If your user is 
too busy to give you specifications or to test the system 
and your deadline approaches, what do you do? If team 
members disagree and can’t get any work completed, 
knowing about the SDLC does not help a bit. 
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Students also learn not be afraid of conflict and how to 
resolve conflict themselves. Only in severe cases will the 
lecturers help to resolve conflict. Lecturers must view 
conflict as a mechanism for learning and not something 
to avoid . 

Very importantly, students learn who they are. In most 
cases students can do much more than they ever dreamt 
possible. Just as importantly they learn their own 
limitations and idiosyncrasies. 

We can only expose information systems students to the 
“real world by letting them experience it to a certain 
extent. One of the best ways is through collaborative 
teamwork in the safety of the learning environment. 
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APPENDIX A 

The size of the system must be between 1,000 and 1,400 size units. Size units are allocated as follows. 



Development Object to Be Counted 


Size Units 
per Object 


Minim um Un its 
per System 


Simple reports 


1 


14 


Complex reports (3 levels of control breaks with totals) 


3 


7 


Simple logical screens (one function, for instance, adding, deleting, updating 
or viewing a client) 


1 


200 


Complex screens (Interlinking multiple functions, for instance, displaying 
master detail) 


3 


50 


Tables in the database 


2 


60 


Table relationships (O ne-to-m any, etc.) 


1 


40 


Interfaces to external devices 


5 


1 


Online document (help, user manual, etc.) parts. (Count every non-repeating 
index and link) 


1 


300 


Non-database files 


2 


4 


OLE interfaces to MS-Word, etc. 


10 


2 
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The system must have at least the following functionality over and above the client’s requirements. 

• The systems cannot only be of a technical nature (e.g., a weather modeling system). At least 80% of the system 
must be of a business nature. (E.g., stock management, banking, retail, creditors, debtors, MRP, manufacturing 
(non-technical side), etc.). 

• Proper security, including encryption of passwords, user profiles, etc. 

• A complete context-sensitive Web-based help file. 

• A complete suite of reports. 

• OLE interfacing to any Windows program directly from the program. E.g., creating a letter to the client in MS- 
Word from information in the database. 

• The use of grids, for multiple rows of data, where appropriate. 

• The use of master-detail table relationships. 

• Maximum flexibility. E.g., a list of branches should not be hard coded but maintained in a sub-module of the 
system. 

• The use of graphs, where appropriate. 

• A totally self-contained installation disk and instructions for the system. This installation must also create the 
ODBC settings needed and an entry in the Windows S tart/Pro gram menu. An uninstall feature must be provided 
that will completely delete all traces of the installed system. 

\ • The storage and display of graphical information, like photos. 

• A link to an accounting package, like Pastel, for the automatic creation of General Ledger batches. 

• A Web-server interface to the system where users can capture and display specific database data. 

• Automatic creation of e-mail messages. 

• The system can’t only be a database maintenance system, the major part of the system must be transaction 
processing; e.g., ordering stock, creating invoices, etc. 

• An audit trail of all transactions in the system. 

• At least one direct interface to an external device (not a printer or a screen). This could be a digital camera, swipe 
card reader, barcode reader, scanner, voice recognition device, etc. 

• User-friendly error messages. System error messages must be intercepted and made user-friendly. 

• Basic interfacing to the W indows system registry. 

• Full access to one or more files in the system (Create, update, read, delete). 
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• At least one function in the system must be timer related. For instance, every hour while the system is running, 
determine the stock levels ofthe system and prompt the user to take appropriate action ifthe stock levels are too 
low. 

• Referential integrity must be enforced. If a user tries to delete a parent record with dependent children records, 
stop deletion or warn and cascade the deletion. 

• A complete easy-to-use backup and restore sub-system. 

• The proper use of standard key strokes, e.g., Ctrl C Copy, Ctrl X Cut, etc. 

The system must be broken up into 3 or 4 (depending on the number of team members) distinct functional parts of 
approxim ately the same size. Every team mem ber must develop one of these parts for the final project demonstration 
and will be evaluated individually. 

The system cannot be developed in database software (e.g., ACCESS reports and forms). A programming language 
such as Delphi or Visual Basic must be used to interface to such databases. 
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A CONCEPTUALIZATION IN M-LEARNING 



Timothy R. Hill 
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Extended Abstract 



Sahai andMachiraju (200 1 ) describe the development of 
Totale-Mobile, Hewlett-Packard’s (HP’s) infrastructure 
for delivering e-services to the heterogeneous mobile 
computing/communications community using a “One 
URL fits all”® approach that “scales” a single web page 
to the requesting device, be it a pager, phone, PDA or 
PC. This convergence enables a host of previously 
inconceivable orinfeasible opportunities both forMobile 
Commerce (m-Commerce) and Mobile Learning, or m- 
Learning, applications. One such application in the m- 
Learning realm is the mobile online class discussion 
conceptualized herein. 

Recently, threaded discussion has been widely adopted 
by the academic community as a collaborative learning 
technique and, despite its inherent limitations, its value 
as such has been well-established. In the 1980’s, the 
Unix-based Usenet and dial up bulletin boards defined 
the first generation of threaded discussion and the geo- 
dispersed community phenomenon was established. 
Then, in the 90's,the availability ofWeb-based, hosted 
(typically free) discussion group services brought a new 
level of accessibility and usability for both students and 
instructors, clearing the way for this second generation 
incarnation to become a mainstay of the emerging 
learner-centric educational pedagogy. 

The benefits are clear and tangible, yet limited still, in 
this second generation, by 1) the fact that, for the most 
part, discussion groups are accessible only via personal 
computers (PC’s) and 2) the passive nature of the 
medium — the burden lies with the discussion 



participants to monitor the discussion site to check for 
new activity. When discussion activity is low, frequent 
checks of the site will disappoint participants, dampen 
their enthusiasm and lead to less frequent checking 
which in turn lowers activity further, feeding a vicious 
cycle toward extended periods of dormancy. Some 
discussion services provide a partial remedy by alerting 
authors, via email, that a reply has been posted, but this 
only improves things incrementally and only if 
participants check email more frequently than the 
discussion site. Without email pagers like the RIM 
Blackberry, participants are still bound by any physical 
location constraints that hamper their connectivity. 

Emerging mobile technologies provide a vehicle for 
evolving threaded discussion to a third generation that 
betteremulates face-to-face discussionsby delivering the 
discourse, in device-scaled form, to the participants in 
real time wherever they are . Thus, the 3G threaded 
discussionis liberated from the desktop and the qualities 
of dynamism and immediacy found in “instant 
messenger” services are extended to the geo-dispersed 
discussion, taking the paradigm from “pull” beyond 
“push,” to a “reach” orientation. In its mobile-enabled 
3G form, threaded discussion will afford a greater sense 
of synchronicity and dynamism as participants are 
apprised of activity as it occurs and, depending on their 
individual device, may be able to respond immediately. 
And 3G threaded discussion interaction will be tailored 
to the capabilities and preferences of individual 
participants though customizable agents to optimize 
effectiveness and to prevent intrusiveness and the 
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annoyance associated with information overload. 
Clearly, providing this kind of highly customizable 
service across the diverse array of connective devices 
found among a typical university-level class is a valuab le 
m-Learning enhancement being enabled by emerging 
mobile technologies such as those developed at HP. 
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Extended Abstract 



Critical-thinking skills in general, and computer- 
comprehension skills in particular, are increasingly seen 
as necessary for the successful accountant as well as for 
the information-systems professional. While other 
universities have developed separate accounting 
information systems programs, we have addressed this 
critical thinking issue by incorporating three existing 
computer information systems (CIS) courses into our 
accounting major, including a beginning programming 
course, an analysis and design course, and a database 
course. We encourage our accounting students also to 
obtain a. CIS minor, which requires an additional 12 
credit hours, or even a second CIS major while working 
toward their 1 50 credit hours and passage of the CPA. 

The inclusion of a required beginning-programming 
course in the accounting major is somewhat 
controversial as it is thought too demanding by some 
aspiring accounting students and they often begin the 



course wondering what programming has to do with 
accountancy. Considerable work has been conducted to 
adapt our beginning programming course to applicable 
to both CIS and Accounting students and we spend 
significant time in the course helping accounting 
students understand that this course will aid them in 
several aspects: First, it will improve their 

understanding of how a computer works and what it can 
do and cannot do. Second, it will improve students’ 
critical and creative thinking skills, and third, it will 
improve their ability to develop and comprehend 
accounting“algorithms”, allowing them, for examp le, to 
locate a “bug” or “embezzle hole” in an accounting 
flowchart. 

Ours is a rigorous beginning programming course 
similar to what is taught in a computer science 
department; generally roughly half of the students who 
start the course receive a grade of “C” or higher. It 
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includes subroutines and functions with pass-by-value 
and pass-by-referenceparameterpassing, sequential file 
access, array-searching algorithms, and an extension of 
the bubble sort to solve a payroll problem. While many 
of the problems focus on business applications, we have 
found that fun graphic projects, such as building “choo 
choo train” screen saver, provide increased student 
motivation while still teaching the essential course 
concepts. Considerable time is spent teaching a 
structured approach to problem solving including 
learning to design carefully algorithms using pseudo 
code and structure charts. Functional cohesiven ess with 
procedures and modularization are emphasized. 

Accounting students have done quite well in this 200 
level course. We examined the grades of all declared 
accountingand CIS students who took this course during 
the 1999 and 2000 school years (35 CIS majors and 31 
accounting majors). On a A = 4 and F = 0 scale, 
declared accounting students and declared CIS majors 
each had an average grade of 2.5 1 . And 68% of declared 
accounting majors had a grade of “C” or better while 
67% of declared CIS majors had agrade of “C” orbetter. 
Other preliminary results indicate that accounting 
students, who have successfully completed the 
beginning-programming course, perform better in our 
300-level advanced accounting course than those who 
have not. 



Further, the influx of roughly 5 0% accounting students 
into the beginning-programming course has had the 
additional positive effect of increased mutual respect 
between accounting and computer information system 
students. The accounting students have come to realize 
thatprogramming is a demanding intellectual activity as 
intense as anything in the accounting program . And CIS 
majors have come to realize that the accounting majors 
can match them at their own game. 

Acceptance of accounting majors in the beginning- 
programming course is on the rise, many accounting 
majors are completing the CIS minor, and a significant 
number are adding a second CIS Major. While our 
approach of adding existing CIS courses to the 
accounting major appears to be a viable alternative to a 
separate AIS curriculum, it success is highly depending 
on several factors: first, on having fairly small class 
sizes; second, on having the support of the college and 
university administration; and third, on having the 
cooperation of the CIS faculty teaching these courses, as 
they must spe nd additional time de veloping and teaching 
these courses when they include accounting and other 
non-traditional programming students. 
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INTRODUCTION 

IS 2002: Model Curriculum and Guidelines for 
Undergraduate Degree Programs in Information Systems 
is the latest report on the model curriculum work in the 
information systems field. The work of IS curricula task 
groups began in the early 1970s and has continued for 
the past 30 years. The Association for Computing 
Machinery (ACM) has been a major organizer for these 
task groups including the first efforts in the 1970s. Other 
organizations, including AIS (Association for Infor- 
mation Systems), AITP (formerly DPMA) and IFIP 
(International Federation for Information Processing), 
have aided model curriculum development. 

IS 2002 is the second collaborative effort between ACM, 
AITP, and AIS. All three organizations have worldwide 
membership. ACM has both professional and academic 
members in the broad field of computing. Through its 
Education Board, it supports a wide range of curriculum 
development including computer science, information 



systems, and software engineering. AITP is a pro- 
fessional organization primarily composed of 
information system practitioners that focuses on 
education and professional development of its members. 
AIS, organized in 1994, is composed of faculty members 
in information systems. The partnership of ACM, AITP, 
and AIS, therefore, combines the breadth of interest of 
ACM, AITP, and the information system interests of 
AIS. 

Although ACM, AITP, and AIS are worldwide 
organizations, IS 2002 does not represent a universal 
curriculum. It does not seek to harmonize the curri- 
culum to meet the requirements of different educational 
systems around the world. The model curriculum for 
undergraduate degree programs in information systems 
is based on the typical degree structure in USA and 
Canadian universities. The model curriculum can, 
however, serve as a useful reference for designers of 
information systems degree programs outside the USA 
and Canada. 
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The most recent undergraduate curriculum model, IS ‘97 
(Davis et al., 1997; Couger et al., 1997) was circulated 
in draft form in 1994 (Gorgone et al., 1994; Longenecker 
et al., 1994) and 1995 (Couger etal., 1995) and finalized 
in 1996. Members of the Joint Task Force presented 
drafts of IS ‘97 at numerous conferences from 1994 to 
1996 and received significant feedback that substantially 
strengthened the report. Additionally, much of the work 
for the 1997 model was actually completed between 
1994 and 1995. Thus, it has been approximately seven 
years since the current model curriculum has been 
updated. Since 1997, the task force members have been 
collecting survey data to better understand who and how 
the model curriculum was being used. Given the changes 
that have occurred since 1995 in the information systems 
discipline, this paper provides a preliminary report on 
the process of updating IS ’97. 

In the next section, we briefly outline the motivation 
behind the need to update IS ’97. This is followed by a 
brief review of the guiding assumptions about the IS 
profession that helped to shape the curriculum design 
and evolution. Next, the scope of the curriculum update 
is presented so that programs transitioning from IS ’97 
to IS 2002 will better understand the model’s evolution. 
This is followed by a brief presentation of the course 
architecture and intended course sequence. Finally, the 
paper concludes by providing high-level course descrip- 
tions of the IS 2002 model curriculum. 

MOTIVATION FOR CURRICULUM UPDATE 

Since the last revision of the undergraduate curriculum 
guidelines, three major factors have spurred the need to 
reexamine and update the existing standard. In parti- 
cular, the advent of the Internet, the changes in student 
computing literacy, and the information accreditation 
movement all motivate the need to update the 
curriculum. In this section, we briefly review each of 
these motivating factors. 

Internet Revolution 

As discussed previously, much of the work in 
developing IS ’97 occurred prior to 1995. During the 
writing of IS ’97 the utility of Web and Internet pro- 
gramming was not yet foreseen, with limited references 
to thin-client programming concepts, Internet protocols 
and internetworking, and other relevant content. 
Although it was known that the impact of these then 
novel concepts could be enormous, it was at the time 



unrealized. In the intervening years the Internet has 
grown to become a major aspect of all IS environments. 

Changes in Student Computing Literacy 

Over the past decade, there has been a significant change 
in the basic computer literacy of incoming university 
students. In the past, very few students entered a 
university having significant skills in using a desktop 
computer, with even fewer students owning or having 
easy access to a computer. Today, with the advent of the 
Internet revolution and low cost PC’s, most students 
entering a university today have at least a modest level 
of computer literacy. 

Information System Accreditation Movement 

There has been interest in the accreditation of programs 
in Information Systems since the accreditation of 
programs in computer science was begun in the mid 
1980s. The work on IS ’97 with its support from the 
major IS professional societies provided much needed 
catalyst for IS accreditation to move forward. With the 
support of the National Science Foundation, the Criteria 
for the Accreditation of Programs in Information 
Systems have been developed with IS ’97 serving as the 
basis of the IS curriculum criteria. ABET is the agency 
with responsibility for accrediting all programs in 
computing, engineering and technology (Gorgone & 
Lidtke, 2000). The Computing Accreditation Commis- 
sion (CAC) has responsibility for accrediting computer 
science and information systems programs. The first 
pilot visit was completed during Fall 2001 . 

GUIDING ASSUMPTIONS 
ABOUT THE IS PROFESSION 

In conceptualizing the role of information systems in the 
future and the requirements for IS curricula, it seems 
apparent that several elements have been (ACM ’72, 
ACM ’82, IS ’90, IS ’97, and ISCC ’99) and remain 
important and characteristic of the discipline. These 
characteristics evolve around three major areas that 
permeate all aspects of the IS profession and therefore 
must be carefully integrated into any IS curriculum: 

1. IS professionals must have a broad business and real 
world perspective. Students must therefore 
understand that: 

• IS enables the success of organizations 



390 



Proceedings of the 16 th Annual Conference of the International Academy for Information Management 



• IS span and integrate all organizational levels and 
business functions 

• IS are increasingly of strategic significance 
because of the scope of organization systems 
involved and the role systems play in enabling 
organizational strategy 

2. IS professionals must have strong analytical and 

critical thinking skills. Students must therefore: 

• Be problem solvers and critical thinkers 

• Use systems concepts for understanding and 
framing problems 

• Understand that a system consists of people, 
procedures, hardware, software and data 

• Be fluent with enterprise data acquisition, 
conversion, transmission, measurement, 
organization, storage, and applicability for 
individuals at all organizational levels 

• Be capable in applying both traditional and newly 
learned concepts and skills 

• Possess skills in understanding and modeling 
organizational process and data, defining and 
implementing technical solutions, managing 
projects, and integrating systems 

• Focus on the application of information 

technology in helping people achieve their goals. 
Organizational productivity is the desired result; 
people are the focus. 

3. IS professionals must have strong interpersonal 

communication and team skills. Students must 

understand that: 

• IS requires collaboration as well as successful 
individual effort 

• IS design and management demands excellent 
communication (oral, written, and listening) 

• IS requires curiosity, creativity, risk taking and a 
tolerance of these abilities in others 



SCOPE OF CURRICULUM UPDATE 

Recently a survey of computing faculty in the United 
States has been conducted to ascertain information on 
two areas. The first was to determine their current view 
of the appropriate depth of mastery for each of the 
elements in the IS ’97 body of knowledge. The second 
was to gather similar information for key skill areas 
identified within IS ’97. Some of the observation of this 
research has been published (Longenecker et al., 2000; 
Landry et al., 2000). The primary conclusions are 
summarized as: 

1. IS analysts have specific skills at approximately IS 
’97 skill depth level 3 (the ability to USE 
knowledge) in areas of Interpersonal and Team 
Skills, Business Knowledge, Organizational Process 
Development (including IS Systems Analysis and 
Design), Project Management, Database, Software 
Development, Web Programming, and Systems 
Integration. 

2. Skills identified in IS ’97 as Exit Curriculum Areas 
match expectations of the computing industry as 
well as IS faculty. 

3. Skill areas produced by programs of Information 
Technology match expectation of Information 
Systems faculty. 

4. The model courses of IS’97 are acceptable to both IS 
and IT faculty. Interestingly, both CS and SE 
faculty also feel IS’97 courses are relevant. 

When analyzing survey data and IS’97 curricula in 
summer of 2001, the missing element in the curriculum 
was a course focusing on Internet-based commerce. At 
most universities, this course has been a popular required 
course for several years and so there was a clear 
discrepancy in the existing model curriculum and what 
was being operationalized in most universities. Thus, a 
new course was added to the model curriculum, IS 
2002.2 -Electronic Business Strategy , Architecture and 
Design , to address this limitation. Without restructuring 
other aspects of the model curriculum, the addition of a 
new Internet-based commerce course to the model 
curriculum would result in curriculum of eleven required 
courses. There was a desire by the committee to limit 
model curriculum to a target of ten courses, or 30 



Proceedings of the 16 th Annual Conference of the International Academy for Information Management 



391 



semester units, which therefore would result in the 
elimination of one of the existing IS ’97 courses. This 
decision was driven by recently approved accreditation 
standards and by the practical credit hour constraints 
realized by many IS programs. 

IS ’ 97 had a prerequisite course, IS '91 .VO-Knowledge 
Work Software Tool Kit , that assumed students had 
elementary exposure to a suite of software tools useful 
for knowledge workers (spreadsheets, databases, 
presentation graphics, database retrieval, statistics, word 
processing, and electronic mail). It was also assumed 
that students could gain this knowledge through a formal 
course or through self-study modules. Beyond this 
course, IS ’ 97 had a required course, IS ’97 2-Personal 
Productivity with IS Technology , that focused on 
improving student skills in using packaged software, in 
both individual and group work, by designing and 
developing solutions. Given the need to limit the IS 
2002 curriculum to ten courses and the rapid and 
significant improvement in the general computing 
literacy of entering students, IS ’97. P0 and IS ’97.2 were 
melded into a single prerequisite course, IS 
2002.P0 -Personal Productivity with IS Technology. 

Beyond the addition of the Internet-based commerce 
course, IS 2002.2, and the melding of IS ’97. P0 and IS 
’97.2 into a single prerequisite course (or self-study 
modules, see below for more details), the remaining 
courses were retained with appropriate updating of the 
course “Scope” and “Topic” descriptions. In most cases, 
this resulted in the addition of Internet centric-content 
and more contemporary terminology and concepts. For 
all but one, course names remained unchanged. 
However, IS ’97 .9-Physical Design and Implementation 
with a Programming Environment was changed to IS 
2002.9-Physical Design and Implementation in 
Emerging Environments. This change was motivated by 
the continual evolution in rapid application development 
and programming environments. A mapping courses and 
a summary of the changes is represented in Table 1. 

COURSE ARCHITECTURE AND SEQUENCE 

The IS 2002 curriculum assumes that students have a 
prerequisite knowledge of desktop computing. Specifi- 
cally, it is assumed that students have an elementary 
exposure to a suite of software applications useful for 
knowledge workers such as word processing, 
spreadsheets, Email, and Internet browsing. In addition, 
students are assumed to also have knowledge and skill of 
IS technology to be a successful knowledge worker as 



described in the prerequisite course IS 2002. PO- 
Personal Productivity with IS Technology. 

Figure 1 shows the course architecture and sequence of 
courses within IS 2002, including the prerequisite course 
IS 2002.P0. The structure is a “suggested” architecture 
and sequence with the appreciation that each university’s 
situation is somewhat unique. In any event, this 
architecture allows the entire program to be completed 
within a scope of two years. This model will therefore fit 
within the broader curricula constraints of must business 
schools. For IS programs housed outside business, great 
flexibility in the sequence can be employed. 

COURSE DESCRIPTIONS 

In this section, we conclude by providing high-level 
course descriptions for IS 2002 which includes the 
prerequisite Personal Productivity with IS Technology 
(IS’02.P0) course plus the ten courses within the Model 
Curriculum. A title, statement of scope, and list of 
typical topics are used to describe each course. 

IS 2002.P0-Personal Productivity with IS Technology 

(Prerequisite: elementary knowledge of word processing, 
spreadsheets, Email, and Internet browsing) 

SCOPE This prerequisite course enables students to 
improve their skills as knowledge workers. 
The emphasis is on personal productivity 
concepts through using functions and features 
in computer software such as databases, 
presentation graphics, and Web authoring. 
Although identified as a course, this material 
can be delivered as self-study modules, as 
modules associated with other courses using 
the software, or as a full course. 

TOPICS Knowledge work productivity concepts; 

advanced software functionality to support 
personal and group productivity such as 
templates and macros; reuse rather than build 
from scratch; organization and management of 
data (sorting, filtering) via spreadsheets and 
database tools; accessing organization and 
external data; information search strategies; 
tool use optimization and personalization; 
professional document design; Web page 
design and publishing; effective presentation 
design and delivery. 
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TABLE 1 

MAPPING OF IS 2002 COURSES TO IS ‘97 COURSES 



IS 2002 Courses IS’97 Courses 



IS’97.PO - Knowledge Work Software Tool Kit 

IS 2002.P0 - Personal Productivity with IS .4 ' 

Technology MELDING IS’97 .2 - Personal Productivity with IS 





^ Technology 


IS 2002. 1 - Fundamentals of Information 
Systems 


UPDATE IS *97. 1 - Fundamentals of Information Systems 

◄ 


IS 2002.2 - Electronic Business Strategy, 
Architecture and Design 


NO EXISTING COURSE 


IS 2002.3 - Information Systems Theory 
and Practice 


UPDATE IS*97.3 - Information Systems Theory and 

^ Practice 


IS 2002.4 - Information Technology 
Hardware and Systems Software 


UPDATE IS*97.4 - Information Technology Hardware 

^ and Systems Software 


IS 2002.5 - Programming, Data, File and 
Object Structures 


UPDATE IS*97.5 - Programming, Data, File and Object 

^ Structures 


IS 2002.6 - Networks and 
Telecommunication 


UPDATE IS’97.6 - Networks and Telecommunication 

◄ 


IS 2002.7 - Analysis and Logical Design 


UPDATE IS’97.7 - Analysis and Logical Design 

◄ 


IS 2002.8 - Physical Design and 
Implementation with DBMS 


UPDATE IS’97.8 - Physical Design and Implementation 

M with DBMS 


IS 2002.9 - Physical Design and 


UPDATE IS’97.9 - Physical Design and Implementation 



Implementation in Emerging Environments ^ with a Programming Environment 



IS 2002. 10 - Project Management and 
Practice 


UPDATE IS*97. 10 — Project Management and Practice 

◄ — : 
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FIGURE 1 

IS 2002 COURSE ARCHITECTURE 



\ 




IS 2002. 1-Fundamentals of Information Systems 

(Prerequisite: IS’02.P0) 

SCOPE This course provides an introduction to 
systems and development concepts, infor- 
mation technology, and application software. 
It explains how information is used in 
organizations and how IT enables 
improvement in quality, timeliness, and 
competitive advantage. 

TOPICS Systems concepts; system components and 
relationships; cost/value and quality of infor- 
mation; competitive advantage and 
information; specification, design and re- 
engineering of information systems; 
application versus system software; package 
software solutions; procedural versus non- 
procedural programming languages; object 



oriented design; database features, functions, 
and architecture; networks and telecom- 
munication systems and applications; 
characteristics of IS professionals and IS 
career paths. Practical exercises may include 
developing macros, designing and 
implementing user interfaces and reports; 
developing a solution using database software. 

IS 2002.2-Electronic Business Strategy, Architecture 

and Design (Prerequisite: IS’02.1) 

SCOPE This course examines the linkage of 
organizational strategy and electronic methods 
of delivering products, services and exchanges 
in inter-organizational, national, and global 
environments. Information technology stra- 
tegy and technological solutions for enabling 
effective business processes within and 
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between organizations in a global environ-ment 
are considered. 

TOPICS Electronic commerce economics, business 
models, value chain analysis, technology 
architectures for electronic business, supply 
chain management, consumer behavior within 
electronic environments, legal and ethical 
issues, information privacy and security, 
transborder dataflows, information accuracy 
and error handling, disaster planning and 
recovery, solution planning, implementation 
and rollout, site design, Internet standards and 
methods, design of solutions for Consumer 
Internets, Intranets and Extranets, EDI, 
payment systems, support for inbound and 
outbound logistics. 

IS 2002.3-Information Systems Theory and Practice 

(Prerequisite: IS’02.1) 

SCOPE This course provides an understanding of 
organizational systems, planning, and decision 
process, and how information is used for 
decision support in organizations. It covers 
quality and decision theory, information theory, 
and practice essential for providing viable 
information to the organization. It outlines the 
concepts of IS for competitive advantage, data 
as a resource, IS and IT planning and imple- 
mentation, change and project management. 

TOPICS Systems theory and concepts; information 
systems and organizational system; decision 
support; quality; level of systems: strategic, 
tactical and operational; system components 
and relationships; information system stra- 
tegies; roles of information and information 
technology; roles of people using, developing 
and managing systems; IS planning and 
change management; human-computer 
interface; IS development process; evaluation 
of system performance; societal and ethical 
issues related to information systems design 
and use. 

IS 2002.4-Information Technology Hardware and 

System Software (Prerequisite: IS’02.1) 

SCOPE This course provides the hardware / system 
software fundamentals for various computer / 
network architectures used in the design, 



development and implementation of contem- 
porary information systems. These concepts 
enable systems development personnel to 
explain tradeoffs in computer architecture for 
effective design. System architecture for 
single user, central, and networked computing 
systems; single and multiuser operating 
systems. 

TOPICS Hardware: CPU architecture, memory, 

registers, addressing modes, busses, 
instruction sets, multi processors versus single 
processors; peripheral devices: hard disks, 
CDs, video display monitors, device 
controllers, input/output; operating systems 
functions and types; operating system 
modules: processes, process management, 
memory and file system management; 
examples and contrasts of hardware archi- 
tectures, operating systems, basic network 
components and telecommunication devices 
for the implementation of information 
systems. 

IS 2002.5-Programming, Data, File and Object 

Structures (Prerequisite: IS’02.1) 

SCOPE This course provides an exposure to algorithm 
development, programming, computer con- 
cepts and the design and application of data 
and file structures. It includes the use of 
logical and physical structures for both 
programs and data. 

TOPICS Data structures and representation: characters, 
records, files, multimedia; precision of data; 
information representation, organization and 
storage; algorithm development; programming 
control structures; program correctness, 
verification, and validation; file structures and 
representation. Programming in traditional 
and visual development environments that 
incorporate event-driven, object-oriented 
design. 

IS 2002.6-Networks and Telecommunication 

(Prerequisite: IS ’02.4) 

SCOPE This course provides an in-depth knowledge 
of data communications and networking 
requirements including networking and tele- 
communications technologies, hardware, and 
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software. Emphasis is upon the analysis and 
design of networking applications in 
organizations. Management of telecom- 
munications networks, cost-benefit analysis, and 
evaluation of connectivity options are also 
covered. Students learn to evaluate, select, and 
implement different communication options 
within an organization. 

TOPICS Telecommunication configurations; network 
and Web applications; distributed systems; 
wired and wireless architectures, topologies 
and protocols; installation, configuration and 
operation of bridges, routers, switches and 
gateways; network performance tuning; 
privacy, security, firewalls, reliability; 
installation and configuration of LAN and 
WAN networks; monitoring of networks; 
management of telecommunications, and 
communications standards. 

IS 2002.7-Analysis and Logical Design (Prerequisite: 
IS’02.1) 

SCOPE This course examines the system development 
and modification process. It emphasizes the 
factors for effective communication and 
integration with users and user systems. It 
encourages interpersonal skill development 
with clients, users, team members, and others 
associated with development, operation and 
maintenance of the system. Structured and 
object oriented analysis and design, use of 
modeling tools, adherence to methodological 
life cycle and project management standards. 

TOPICS Life cycle phases: requirements determina- 

tion, logical design, physical design and 
implementation planning; interpersonal skills, 
interviewing, presentation skills; group 
dynamics; risk and feasibility analysis; group- 
based approaches: project management, joint 
application development (JAD), structured 
walkthroughs; structured versus object 
oriented methodologies; prototyping; database 
design; software package evaluation, 
acquisition, and integration; global and inter- 
organizational issues and system integration; 
professional code of ethics. 



IS 2002.8-Physical Design and Implementation with 

DBMS (Prerequisite: IS’02.5 and IS’02.7) 

SCOPE This course covers information systems 
design and implementation within a database 
management system environment. Students 
will demonstrate their mastery of the design 
process acquired in earlier courses by 
designing and constructing a physical system 
using database software to implement the 
logical design. 

TOPICS Conceptual, logical and physical data models 
and modeling tools; structured and object 
design approaches; models for databases: 
relational, hierarchical, networked and object 
oriented; design tools; data dictionaries, 
repositories, warehousing and data mining; 
database implementation including user 
interface and reports; multi-tier planning and 
implementation; data conversion and post 
implementation review. 

IS 2002.9-Physical Design and Implementation in 

Emerging Environments (Prerequisites: IS’02.2 and 
IS’02.8) 

SCOPE This course covers physical design and 
implementation of information systems 
applications. Implementation in emerging 
distributed computing environments using 
traditional and contemporary development 
environments. 

TOPICS Selection of development environments and 
standards; software construction: structured, 
event driven and object oriented application 
design; testing; software quality assurance; 
system implementation; user training; system 
delivery; post implementation review; con- 
figuration management; maintenance. Multi- 
tier architectures and client independent 
design. 

IS 2002.10-Project Management and Practice 

(Prerequisite: IS’02.7) 

SCOPE This course covers the factors necessary for 
successful management of information 
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systems development or enhancement 
projects. Both technical and behavioral 
aspects of project management are applied 
within the context of an information systems 
development project. 

TOPICS Managing the system life cycle: requirements 
determination, design, implementation; 
system and database integration issues; net- 
work management; project tracking, metrics, 
and system performance evaluation; managing 
expectations of managers, clients, team 
members and others; determining skill 
requirements and staffing; cost-effectiveness 
analysis; reporting and presentation 
techniques; management of behavioral and 
technical aspects of the project; change 
management. Software tools for project 
tracking and monitoring. Team collaboration 
techniques and tools. 
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